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PREFACE 

The  Open  Systems  Acquisition  &  Supportability  Guide  summarizes  and  provides  information  gained 
through  both  the  Navy’s  Next  Generation  Computer  Resources  (NGCR)  program  and  early  Navy  users  of 
commercially  based  open  systems.  The  primary  goal  is  to  help  Navy  programs  to  plan  and  implement  open 
systems  in  a  cost  effective  manner.  When  the  NGCR  program  was  initially  founded,  many  people  said  “we  will 
never  use  open  standards  for  warfare  systems”.  However,  open  standards  usage  is  current  DoD  policy  and  program 
offices  are  learning  the  benefits  of  applying  an  open  systems  approach  to  achieve  an  architectural  foundation  for 
system  upgrade  and  development.  Discussions  continue  as  to  definitions  for  specific  terms  but  overall  open 
systems  concepts  and  goals  are  now  widely  accepted. 

The  information  base  provided  by  this  Guide  is  not  meant  to  be  all  inclusive.  Rather,  it  is  meant  to 
provide  a  starting  point  for  program  specific  planning.  Tailoring  and  adding  to  the  questions  and  concepts 
presented  in  this  Guide  is  appropriate  and  recommended  to  identify  and  meet  external  and  internal  program  needs. 
Experience  will  add  to  the  information  on  open  systems  and  provide  more  data  for  future  acquisition  and  support 
planning. 

The  initial  draft  of  the  Open  Systems  Computer  Resources  Acquisition  Guide  was  published  in  April 
1995.  The  draft  of  the  Open  Systems  Computer  Resources  Supportability  Guide  was  published  in  September  1995. 
Both  documents  were  well  received  and  used  as  foundation  material  for  program  planning  of  efforts  to  develop, 
upgrade,  and  support  Navy  and  other  mission  critical  government  systems.  This  Guide  combines,  clarifies,  and 
expands  the  information  of  both  of  these  documents. 

It  is  time  to  move  on  and  to  thank  those  hardy  souls  who  first  saw  the  light  and  helped  the  rest  of  us  to  do 
so.  Participants  in  the  development  of  this  document  are  too  numerous  to  list  here.  Contributions  came  from 
people  from  the  Department  of  the  Navy,  other  Department  of  Defense  (DoD)  elements,  industry,  and  academia. 
Some  people  and  organizations  participated  in  development  of  the  initial  draft  Open  Systems  Computer  Resources 
Acquisition  Guide.  Other  people  participated  in  development  of  the  Open  Systems  Computer  Resources 
Supportability  Guide.  Particular  thanks  are  hereby  expressed  to  Karl  McClure,  Larry  Holtzman,  and  Mark 
Chestnutwood  at  the  Naval  Surface  Warfare  Center,  Crane  Indiana;  to  Don  Ross  at  the  Naval  Air  Warfare  Center  - 
Aircraft  Division,  Indianapolis;  and  to  John  Brinkheide,  Cari  Dorman,  and  Dale  Anderson  from  Computer 
Sciences  Corporation.  This  group  contributed  their  impressive  weapons  system,  C4I,  systems  engineering,  and 
other  appropriate  experience  bases.  They  dissected  and  reassembled  relevant  information  into  what  we  hope  is  a 
useful  tool  for  program  offices  and  the  organizations  which  support  them.  Thanks  to  all  of  you. 


Alexander  Lewin 
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1.0  INTRODUCTION 

1.1  Purpose  and  Audience 

The  purpose  of  this  Guide  is  to  provide  program  managers,  their  staffs,  supporting  warfare  centers, 
laboratories,  and  contractors  with  a  framework  of  open  system  acquisition  and  supportability  planning  knowledge. 
This  knowledge,  tailored  for  specific  program  usage,  will  enable  them  to: 

•  identify  and  address  issues  and  considerations  which  are  unique  or  more  prevalent  in  an  open  system 
acquisition,  and 

•  work  together  comfortably  and  cost  effectively  to  transition  from  use  of  Department  of  Defense  (DoD) 
unique  interfaces  to  commercially  based  open  architectures. 

1.2  Scope 

This  Guide  addresses  many,  but  certainly  not  all,  of  the  issues  present  in  moving  Navy  weapon  and 
Command,  Control,  Communications,  Computer,  Intelligence,  Surveillance,  and  Reconnaissance  (C4ISR)  systems 
from  proprietary  closed  architectures  to  open  interface  standards  based  solutions  and  their  respective  acquisition 
and  supportability  needs. 

Also  addressed  are  many  of  the  issues  which  stem  from  the  need  for  Joint  operations  and  interoperability. 
Requirements  are  addressed  in  directives  such  as:  DoDD  4630.5,  Compatibility,  Interoperability,  and  Integration  of 
C3I  Systems,  dated  12  Nov  92,  which  stipulates  that  “...all  C3I  systems  developed  for  use  by  U.S.  forces  are 
considered  to  be  for  Joint  use.”  and  DoDI  4630.8,  Procedures  for  Compatibility,  Interoperability,  and  Integration  of 
C3I  Systems,  dated  18  Nov  92,  which  states  that  “...DISA,  shall  certify . that  C3I  systems  and  equipment  meet  the 


This  Guide  uses  short  narrative  descriptions,  figures,  tables,  process,  and  topic  oriented  discussions  to 

provide: 

•  important  open  systems  terms,  concepts,  and  major  transition  issues  to  be  considered  when  acquiring 
open  systems, 

•  an  acquisition/systems  engineering  process  oriented  view, 

•  topic  oriented  discussions  of  supportability  functions,  and 

•  topic  oriented  discussions  of  various  tools  and  methods. 

1.3  Source 

The  information  presented  in  this  Guide  was  gathered  from  various  Navy  programs  which  are  upgrading 
legacy  systems  or  creating  new  systems  on  an  architectural  foundation  of  commercial  open  interface  standards. 
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2.0  UNDERSTANDING  OPEN  SYSTEMS 

Procurement  of  open  system  designs  for  DoD  system  applications  represents  a  significant  change  in  DoD 
acquisition  and  support  methodology.  Designing  military  systems  based  on  open  architectures  and  interface 
standards  makes  economic  sense  for  DoD  in  today's  environment  of  rapidly  evolving  commercially-based 
technologies  and  declining  budgets.  The  need  to  upgrade  systems  in  a  modular,  cost-effective,  best  value  manner 
has  never  been  greater.  Using  widely-accepted  commercial  interface  standards  as  opposed  to  standards  unique  to 
the  military  provides  the  added  cost-avoidance  benefits  of  leveraging  a  large  market  base. 

The  open  systems  approach  mandated  by  DoD  policy  encompasses  the  selection  of  specifications  and 
standards  adopted  by  industry  standards  bodies  or  de  facto  standards  for  selected  system  interfaces,  products, 
practices,  and  tools.  Open  systems  standards  define  interfaces  which  support  portability,  interoperability,  and 
scaleability  (i.e.,  expansion);  and  that  are  available  to  the  public.  An  open  systems  standard  is  primarily  concerned 
with  interface  compatibility.  The  use  of  open  system  interface  standards: 

•  promotes  interoperability  between  multiple  suppliers'  equipment, 

•  does  not  imply  or  assure  the  existence  of  interchangeable  products  from  different  sources, 

•  promotes  both  initial  competition  among  products  and  competitive  upgrade  opportunities;  thus  the  ability 

to  reduce  costs  though  competition  is  promoted,  and 

•  promotes  but  is  not  synonymous  with  commercial  item  products. 

To  gain  advantages  from  open  standards  usage,  it  is  imperative  that  program  management  offices  have  an 
understanding  of  the  objectives,  requirements,  and  approaches  necessary  to  achieve  an  open  system  environment 
(OSE).  This  section  of  the  Guide  contains  discussions  of: 

•  open  systems  concepts  and  terms, 

•  standards  and  standards  bodies,  and 

•  major  transition  issues,  such  as 

•  DoD  initiatives  and  requirements, 

•  risk  management, 

•  quality  assurance, 

•  software  engineering. 

Some  key  system  objectives  of  an  OSE  are  listed  below. 


KEY  OSE  OBJECTIVES 

S  Reduction  of  Life  Cycle  Costs  (LCC),  especially  supportability  and  maintainability 
S  Interoperability  between  two  or  more  systems  or  products 
Sharing  of  common  application  support  resources 
■S  Portability 

•S  Optimum  reuse  of  hardware  and  software  components 
•S  Elimination  of  interface  uniqueness 
Plug  and  play  environment 

•S  Elimination  of  redundant  procurement  of  functional  capabilities 
Increase  sources  of  supply  and  vendor  competition 
Improve  technology  insertion  opportunities 
•S  Provide  stable  upgrade  paths  for  technology  and  modularity  of  products 


2.1  Definitions 
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The  term  open  system  has  many  definitions  and  interpretations  (table  2-1).  Though  the  various 
definitions  have  common  elements,  no  formal  agreement  on  any  one  definition  exists.  This  section  discusses  the 
main  factors  of  several  of  these  definitions,  along  with  the  notion  of  standards  and  standards  bodies  that  together 
support  an  OSE. 


OPEN  SYSTEM 

SOURCE 

DEFINITION 

Open  Systems  Joint  Task 

Force  (OSJTF)  and  Technical 
Architecture  Framework  for 
Information  Management 
(TAFIM) 

A  system  that  implements  sufficient  open  specifications  for  interfaces,  services,  and  supporting  formats  to  enable 
properly  engineered  applications  software:  (a)  to  be  ported  with  minimal  changes  across  a  wide  range  of  systems, 

(b)  to  interoperate  with  other  applications  on  local  and  remote  systems,  and  (c)  to  interact  with  users  in  a  style  that 
facilitates  user  portability.  [IEEE  PI  003.0/D  15] 

IEEE  PI 003.0, 

Draft  16  (POSIX) 

“A  system  that  implements  sufficient  open  specifications  or  standards  for  interfaces,  services  and  supporting  formats 
to  enable  properly  engineered  applications  software: 

•  To  be  ported  with  minimal  changes  across  a  wide  range  of  systems 

•  To  interoperate  with  other  applications  on  local  and  remote  systems 

To  interact  with  people  in  a  style  that  facilitates  user  portability.” 

NGCR  and  Tri-Service 

Open  System  Architecture 
Working  Group 

“A  system  that  implements  sufficient  open  specifications  for  interfaces,  services,  and  support  formats  to  enable 
properly  engineered  components  to  be  utilized  across  a  wide  range  of  systems  with  minimal  changes  to  interoperate 
with  other  components  on  local  and  remote  systems,  and  to  interact  with  users  in  a  style  which  facilitates 
portability.  An  open  system  is  characterized  by  the  following: 

•  Well  defined,  widely  used  non-proprietary  interfaces/protocols, 

•  Use  of  standards  which  are  developed/adopted  by  industrially  recognized  standards  bodies 

•  Definition  of  all  aspects  of  system  interfaces  to  facilitate  new  or  additional  systems  capabilities 
for  a  wide  range  of  applications 

•  Explicit  provision  for  expansion  or  upgrading  through  the  incorporation  of  additional  or  higher 
performance  elements  with  minimal  impact  on  the  system.” 

Software  Engineering  Institute 
(SEI)  at 

Camegie-Mellon 

University 

“  An  open  system  is: 

•  A  collection  of  interacting  software,  hardware,  and  human  components 

•  Deigned  to  satisfy  stated  needs 

•  With  the  interface  specification  of  components 

•  Fully  defined  and  available  to  the  public 

•  Maintained  according  to  group  consensus,  and 

•  In  which  the  implementations  of  components  are  conformant  to  the  specification.” 

Faulkner  Information  Services 

“  a  set  of  standards  that  enable  users  to  select  system  and  network  components  from  a  broad  range  of  suppliers  to 
suit  individual  application  requirements  while  preserving  a  homogeneous  information  system  infrastructure.” 

Gary  Nutt,  Open  Systems 

“...components  and  their  composition  are  specified  in  a  non-proprietary  environment,  enabling  competing 
organizations  to  use  these  standard  components  to  build  competitive  systems.” 

Pamela  Gray,  Open  Systems, 

A  Business  Strategy  for  the 
1990s 

“When  the  three  characteristics:  portability,  scaleability,  and  interoperability,  are  taken  together,  and  international 
standards  set  for  them  by  an  open  process,  in  which  anyone  may  participate,  and  the  results  are  available  on  equal 
terms  to  all,  the  result  is  to  define  that  part  of  the  computer  industry  known  as  ‘open  systems’.” 

Table  2-1  Open  System  Definitions 


Each  definition  in  table  2-1  strikes  a  particular  group  as  correct  or  incorrect,  complete  or  incomplete. 
However,  the  key  commonalties  in  most  open  system  definitions  are  producer  independent  (hereafter  referred  to  as 
original  equipment  manufacturer  (OEM)),  non-proprietary,  publicly  available,  and  widely  accepted.  Ideally,  open 
systems  represent  a  transparent  environment  in  which  users  can  intermix  hardware,  software,  and  networks  of 
different  vintages  from  different  OEMs  to  meet  differing  needs.  Other  properties  exhibited  in  an  open  system  are: 

•  Interchangeable  -  (1)  Capable  of  being  interchanged;  esp. :  permitting  mutual  substitution  <~  parts>. 
[Webster's  Dictionary]  (2)  The  ability  of  two  or  more  products  (hardware  or  software)  to  be  transparent 
replacements  for  one  another  without  other  hardware,  software,  firmware,  or  wiring  changes,  [this  Guide] 

•  Interoperability  -  ( 1 )  The  ability  of  the  systems,  units,  or  forces  to  provide  and  receive  services  from  other 
systems,  units,  or  forces,  and  to  use  the  services  so  interchanged  to  enable  them  to  operate  effectively 
together.  [Joint  Pub  1  -02,  DoD/NATO]  [JOPES  ROC]  [TAFIM] 
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•  Portability  -  (1)  The  ability  of  two  or  more  systems  or  components  to  exchange  and  use  information. 
[IEEE  STD  610.12]  (2)  The  ease  in  which  a  system  or  component  can  be  transferred  from  one  hardware 
or  software  environment  to  another.  [IEEE  Std  610.12]  (3)  A  quality  metric  that  can  be  used  to  measure 
the  relative  effort  to  transport  the  software  for  use  in  another  environment  or  to  convert  software  for  use  in 
another  operating  environment,  hardware  configuration,  or  software  system  environment.  [IEEE 
TUTOR1  (4)  The  ease  with  which  a  system,  component,  data,  or  user  can  be  transferred  from  one 
hardware  or  software  environment  to  another.  [TAFIM] 

•  Scaleability  -  (1)  The  ability  to  use  the  same  application  software  on  many  different  classes  of 
hardware/software  platforms  from  personal  computers  to  super  computers  (extends  the  portability 
concept).  (USAICn)  The  capability  to  grow  to  accommodate  increased  loads.  [TAFIM] 

In  reality,  systems  are  not  purely  open  or  closed.  Because  industry  standards  do  not  generally  meet  all 
military  needs,  trade-offs  must  be  made  between  performance,  cost,  supportability,  availability  of  standards  based 
products,  and  the  ability  to  upgrade.  The  result  is  that  for  any  given  system,  the  degree  of  openness  may  have 
many  interpretations. 

2.2  Standards  and  Standards  Bodies 

Open  system  interface  standards  are  the  infrastructure  of  open  systems  design,  the  building  blocks  of  an 
OSE.  They  support  the  OSE  as  they  codify  open  system  ideals.  Standards  are  essential  if  the  desirable  features  of 
open  systems  are  to  be  achieved.  Standards  help  provide  consistency  and  compatibility  within,  across,  and  between 
OEM  products.  An  open  system  interface  standard  is  an  agreed-upon  specification,  identifying  services  and 
protocols,  and  where  appropriate,  mechanical  form  factors. 

The  Institute  of  Electrical  and  Electronics  Engineers  (IEEE)  defines  a  standard  as  “A  document, 
established  by  consensus  and  approved  by  an  accredited  standards  development  organization,  that  provides,  for 
common  and  repeated  use,  rules,  guidelines  or  characteristics  for  activities  or  their  results,  aimed  at  the 
achievement  of  the  optimum  degree  of  order  and  consistency  in  a  given  context.”,  (IEEE  1003.0  Draft  16,  page 
19).  Accreditation  is  a  function  of  the  international  organization  for  standards  (ISO). 

In  the  choice  of  standards  for  consideration,  the  objective  is  to  select  standards  at  as  high  a  level  in  the 
standards  hierarchy  as  possible.  National  and  international  standards  are  developed  in  open  forums  where  users 
and  providers  reach  consensus  concerning  specifications.  Consortia  standards  are  usually  developed  by  a  group  of 
product  manufacturers  on  a  majority  rule  basis.  The  order  of  precedence  for  selecting  interface  standards  is  shown 
in  table  2-2. 


STANDARDIZATION  LEVEL 

STANDARDS  BODIES 

International 

International  Telegraph  and  Telephone  Consultative  Committee  (CCITT) 
International  Organization  for  Standards  (ISO) 

US  National 

American  National  Standards  Institute  (ANSI) 

American  Society  for  Testing  and  Materials  (ASTM) 

Institute  of  Electrical  and  Electronics  Engineers  (IEEE) 

Consortia 

Open  Software  Foundation 

Structured  query  language  (SQL)  Access 

X/Open 

Asynchronous  Transfer  Mode  (ATM)  Forum 

Table  2-2  Order  of  Precedence  for  Standard  Selection 

Other  types  of  standards  include  Federal  Information  Processing  Standards  (FIPS)  published  by  the 
National  Institute  of  Standards  and  Technology  (NIST),  various  government  standards,  military  standards,  and  de 
facto  standards.  De  facto  standards  may  be  proprietary  or  open  but  are  not  recognized  by  a  standards  body. 
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However,  due  to  widespread  use,  de  facto  standards  have  become  accepted  in  industry  as  a  standard.  Examples  of 
de  facto  standards  are:  Microsoft  MS-DOS,  IBM  Systems  Application  Architecture  (SAA),  and  Adobe  Postscript. 

2.2. 1  Standards  Features  and  Profiles 

Each  standard  consists  of  mandatory,  optional,  and  conditional  features  that  are  typically  put  together  in 
functional  groups  referred  to  as  a  standard's  profile.  The  commercial  market  place  usually  aligns  itself  with  a 
standards  profile  (rarely  more  than  one)  in  a  given  product  market.  Product  implementations  reflect  the  industry 
aligned  profile  of  interface  features.1  A  system  is  comprised  of  various  standard(s)  and  associated  profiles 
integrated  to  form  a  set  of  functions  that  achieve  system  requirements. 

In  order  for  the  interfaces  to  function  efficiently,  the  optional  features  of  each  of  the  interfaces  must  be 
aligned  (the  same  or  at  least  have  a  common  subset).  This  creates  the  need  to  document  the  features  implemented 
for  each  of  the  interfaces  in  a  standard!  s)  profile.  The  interfaces  used  by  a  system  regardless  of  where  that  system 
falls  in  the  hierarchy  (figure  2-1)  need  to  be  profiled  to  fully  define  the  data  structures,  protocols  and  physical 
characteristics. 


1)  JOINT  ARCHITECTURES 

EXTERNAL  INTERFACES 


2)  SERVICE  ARCHITECTURES  -  (one  service) 

EXTERNAL  &  INTERNAL  INTERFACES  (COMM  BETWEEN 
SAME  SERVICE  PLATFORMS) 

3)  PLATFORM  ARCHITECTURES  -  (same  service) 

EXTERNAL  &  INTERNAL  INTERFACES  (  COMM  BETWEEN 
SAME  PLATFORM  PROGRAMS 

4)  PROGRAM  ARCHITECTURE  (one  platform) 

EXTERNAL  &  INTERNAL  INTERFACES  (  COMM  BETWEEN 
SAME  SUB-PROGRAMS) 

5)  SUB  PROGRAM  ARCHITECTURE 

EXTERNAL  &  INTERFACES  (  COMM  BETWEEN  SAME 
SUB-PROGRAMS) 


Figure  2-1  Interfaces  to  External  and  Internal  Requirements 


Figure  2-1  illustrates  how  interface  requirements  relate  to  each  other  depending  on  where  the  system  is  in 
the  system-of-systems  hierarchy.  At  the  Joint  level,  external  interfaces  between  Service  systems  and  products  need 
to  be  identified  and  assessed  to  ensure  Joint  Interoperability  requirements  are  addressed  and  satisfied. 

At  the  individual  Service  level  (e.g.,  US  Navy),  external  and  internal  (Service  unique)  interface 
requirements  are  defined  (profiled)  to  achieve  interoperability. 

As  the  requirements  are  defined  for  individual  platforms,  considerations  of  interfaces  can  again  be  seen  at 
the  external  and  internal  levels.  The  external  interfaces  address  system  architecture  requirements  from  platform  to 
platform;  internal  interfaces  address  system  to  system  (and  equipment)  interface  requirements. 

At  the  equipment  level,  the  interfaces  can  again  be  considered  as  external  and  internal.  This  process  and 
interface  consideration  continues  downward  until  there’s  no  interface  to  define  and  profile,  at  the  configuration 
item. 


1  Refer  to  Internet  http://xystar.crane.navy.mil  for  more  information  on  standards  and  profiles. 
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Profiling  of  interface  standards  is  critical  to  the  whole  process.  In  this  instance  profiling  means  not  only 
the  listing  of  interface  standards  but  further  delineation  of  how  these  standards  are  applied  to  the  design  of  the 
system  or  product  being  acquired  to  meet  the  requirements.  It’s  one  thing  to  identify  what  standard  is  needed  to 
obtain  interoperability  but  it's  more  difficult  to  come  to  agreement  what  part  of  the  standard  needs  to  be 
implemented  to  get  to  interoperability  in  the  most  cost  effective  manner  to  all. 

The  need  for  documented  commonality  of  interfaces  is  rooted  in  the  concept  of  Joint  operations.  The  concepts  of 
“Joint  interfaces”"  and  “system-of-systems”  refer  to  the  interfaces  common  between  systems  in  different  Services  (i.e.. 
Army,  Air  Force,  Navy,  and  Marines)  and  a  system  made  up  of  stand  alone  sub-systems  respectively.  This  requirement 
generates  the  need  for  common  data  definitions,  data  formats  and  symbology,  as  well  as  to  transmit,  receive  and  present 
this  data  across  Services,  platforms,  and  other  organizational  lines.  One  way  of  visualizing  the  interfaces  involved  in 
these  concepts  is  that  Joint  interfaces  communicate  between  the  Services,  Service  unique  interfaces  communicate  between 
the  same  Service’s  platforms,  and  system  interfaces  communicate  between  the  sub-system(s)  as  shown  in  figure  2-2.  The 
interfaces  required  at  the  Joint  level  only  need  be  present  on  the  Service  interface(s)  to  the  Joint  interface.  There 
are  many  standard  interfaces  that  make  up  the  full  list  of  Joint  interfaces.  Likewise  the  interfaces  required  at  the 
Service  level  only  need  be  present  on  the  system  interface(s)  to  the  Service  interface.  Thus  the  interfaces  required 
for  Joint  interoperability  among  the  Services  are  not  necessarily  universally  common  across  all  Service  platforms 
and  systems. 


Joint  Interfaces 
Service  Unique 


Figure  2-2  Interface  Relationships 

2.3.  Advantages  and  Considerations 

In  the  past,  DoD  suppliers  (major  contractors)  built  the  majority  of  the  parts  they  supplied  in  DoD 
systems.  Now,  they  buy  open  interface  standards  products  from  a  range  of  other  suppliers,  including  competitors. 


"  Refer  to  paragraph  2.5.2  for  more  information  on  Joint  architectures. 
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integrating  them  into  larger  scale  assemblies  (cards,  boxes,  systems)  for  sale  to  the  government  just  as  to  industry. 
Interface  commonality  and  stability  makes  this  integration  possible.  The  change  to  open  systems  means  DoD  is 
becoming  a  participant  in  the  worldwide  market  for  standards  based  products. 

In  an  open  system  environment,  time  and  cost  for  system  design,  development,  and  integration  can  be 
reduced  through  the  use  of  non-developmental  items  (NDI)  and  commercial  items.  Other  advantages  include: 
increased  system  flexibility;  lower  acquisition  costs;  rapid  prototyping  and  fielding;  and  if  properly  planned  and 
managed,  lower  life  cycle  costs  due  to  the  ability  to  leverage  a  competitive  commercial  market.  Additionally, 
system  upgrades  can  be  made  incrementally  and  without  the  need  for  a  major  system  redesign. 

2.3.1  The  Competitive  Market 

Open  systems  provide  many  advantages  to  users,  system  managers,  and  OEMs.  DoD  now  has  the 
potential  to  mix  and  match  competitive  commercial  item  products  from  many  commercial  sources  that  may  be 
available  in  an  open  system  environment;  thereby,  creating  an  OEM-neutral  environment.  From  an  acquisition 
view  point,  more  product  sources  implies  competition,  resulting  in  decreased  prices  and  increased  performance  and 
OEM  support  services.  Competition  also  offers  the  possibility  of  a  best-value  choice  for  a  given  system  acquisition. 
An  OSE  permits  support  costs  to  be  leveraged  from  the  commercial  marketplace.  This  is  accomplished  by  using 
commercially  based  support  products  and  services,  supplemented  only  as  necessary.  Typical  commercial  support 
includes:  technical  consultation;  technical  documentation;  upgrade  and  repair  of  hardware;  software  maintenance; 
and  training. 

The  benefits  of  open  systems  are  not  limited  to  large  OEMs  capable  of  taking  advantage  of  economies  of 
scale.  Any  OEM  can  build  systems  from  lowest  replaceable  units  (LRU)  marketed  by  any  other  OEM.  Thereby, 
OEMs  are  more  competitive,  yet  cooperative,  among  themselves,  encouraging  higher  quality  products  and  services. 
Unlike  proprietary  system  acquisition,  users  no  longer  must  rely  solely  on  the  OEM  for  sources  of  compatible 
products  for  system  support,  expansion,  upgrades  or  replacement  parts. 

Open  system  interfaces  allow  applications  (software)  to  be  decoupled  from  supporting  hardware. 
Applications  can  then  be  developed  separately  and  more  easily  migrated  among  differing  hardware.  Additionally, 
hardware  changes  do  not  necessarily  force  costly  code  changes  in  applications  because  use  of  standards  achieves 
commonality  at  the  interface  level. 

2.3.2  Benefits  and  Challenges 

The  above  described  benefits  are  not  without  challenges.  Challenges  arise  from  the  use  of  standards 
themselves.  Ambiguities  exist  within  many  standards,  resulting  in  contractors  offering  incompatible 
implementations  of  the  same  interface  standards.  Sparing  commercial  items  may  be  unsuitable  for  some  military 
environments.  Long  term  spares  support  of  commercial  items  is  subject  to  the  vagaries  of  the  commercial 
marketplace,  where  here-today-gone-tomorrow  is  a  common  state  of  affairs. 

Some  of  the  acquisition  benefits  and  challenges  attributable  to  an  open  systems  environment  are  listed  in 
table  2-3.  The  point  to  be  realized  from  table  2-3  is  that  the  availability  of  products  and  existing  services  far  out 
weighs  the  negative  aspects  of  overcoming  any  of  the  challenges  (challenges  may  not  be  disadvantages). 


BENEFITS 

CHALLENGES 

•  Availability  of  current  technology  products  for  initial 
design/productions,  support,  and  upgrade. 

•  Low  unit  cost  for  products. 

•  Keep  modernizing  our  systems  or  our  adversaries  will 
have  more  capability  than  us. 

•  Our  adversaries,  real  and  potential,  have  access  to 
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BENEFITS 

CHALLENGES 

•  Ability  to  leverage  commercial  support,  not  invest  to 
create  and  maintain  it  ourselves. 

•  Support  through  upgrade  yields  better  and  better 
performance  over  years  of  system  use. 

•  More  knowledge  of  products  and  their  total  costs 
(includes  support)  prior  to  committing  to  their  use. 

•  Existing  products  that  can: 

•  Reduce  procurement  lead  time. 

•  Provide  existing  support  structure(s). 

•  Provide  planned  product  improvements  by  the 

OEM. 

•  Have  an  established  upgrade  path. 

•  Have  known  (or  derivable)  costs. 

the  same  technologies  we  do. 

•  Need  to  structure  contracts  to  match  marketplace 
changes  with  respect  to  product  availability  and  price 
changes. 

•  Reduce  contracting  lead  time. 

•  Keep  up  with  what  is  going  on  in  commercial  markets, 
e.g.,  ongoing  market  survey,  test,  and  evaluation. 

•  Handling  higher  configuration  management  workload. 

•  Manage  changing  technologies’  affect  on  maintenance 
costs. 

•  Accept  that  standardization  process  is  slow  compared  to 
the  evolution  of  technology  (state  of  the  art) 

Table  2-3  Open  System  Acquisition  Benefits  and  Challenges 

2.3.3  Influences  on  Supportability 

An  OSE  influences  how  we  provide  for  supportability.  Despite  the  many  considerations  for  supportability 
due  to  an  OSE,  there  are  also  many  potential  benefits.  Table  2-4  depicts  several  supportability  benefits  and 
considerations. 


BENEFITS 

DISCIPLINES 

CONSIDERATIONS 

Lower  support  acquisition  cost. 

Leverage  the  commercial  investment. 

Earlier  availability. 

General 

Supportability 

Rapid  incremental  upgrade. 

Lack  of  Navy  control. 

ILS  products  not  to  Navy  formats. 

Earlier  Planning  data. 

Existing  support  systems. 

Upgrade  in  lieu  of  repair. 

Maintenance  Planning 

Schedule  compression. 

Repair  turn-around  time. 

Upgrade  in  lieu  of  repair. 

Potential  for  fewer  Navy  Enlisted 

Classification  (NEC)  types. 

Potential  for  fewer  personnel. 

Manpower  and  Personnel 

Just  in  time  approach. 

Increased  initial  competition. 

Potential  increased  commonality. 

Supply  Support 

Changing  products. 

Product  configuration  stability. 

Standard  interface  analyzers. 

Commonality. 

Support  Equipment 

Normal  NDI  issues 

Lower  cost. 

Early  availability. 

Technical  Data 

No  control  (changes,  format,  media). 

Data  rights  ownership. 

Licenses  and  royalty  agreements. 

Commercial  training. 

Potential  for  less  intensive  training. 

Training  and  Training 
Support 

Course  content  control. 

Potential  for  more  frequent  training. 

Earlier  “ilities”  data  available. 

Facilitates  proactive  support  team 

Ease  of  upgrade. 

Design  Interface 

Changing  product  families. 

Design  is  not  build-to  print. 

Interchangeability  matrix  concept. 

Configuration  Management 

No  Navy  control. 

Rapid  NDI  product  turnover. 

Other  normal  NDI  issues. 

Table  2-4  Common  Supportability  Benefits  and  Considerations  in  Using  Open  System  Interface  Standard 
Products  Not  Developed  Specifically  for  the  Government 

If  the  government  initially  procures  an  NDI/commercial  item  product  and  does  not  pay  for  its  design, 
getting  necessary  documentation  of  any  unique  design  features  (to  enable  reproduction  or  emulation)  may  be  costly 
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or  impossible.  Even  if  an  initial  design  was  paid  for  by  the  government,  reproduction  of  design  features  still  may 
be  difficult.  Documentation  and  manufacturing  process  differences  among  design/production  houses  drive  this 
issue. 


The  commercial  marketplace  maintains  configuration  control,  technical  data  control  and  support  control 
of  the  products.  Even  if  the  OEM  and  his  competitors  are  fully  engaged  in  the  given  market  segment  and  the 
system  has  no  unique  interface  features,  the  government  or  system  integrator  does  not  have  the  ability  to  control 
the  support  environment  as  it  used  to  do. 

The  market  place  will  continue  to  upgrade  product  lines  while  abandoning  specific  obsolete  products  as 
driven  by  market  forces,  both  military  and  commercial.  Commercial  support  must  be  continuously  monitored. 
Contingency  plans  must  be  developed  to  handle  discontinued  commercial  product  support. 

2.3.4  The  Challenges  of  Non-Conformance 

The  magnitude  of  the  challenges  increase  when  products  being  procured  (NDI/commercial  item  or 
developmental)  do  not  fully  conform  to  interface  standards  and  profiles.  Acceptance  of  proposed  extensions  to  or 
deviations  from  standards  to  achieve  desirable  performance  advantages  often  leads  to  sole  source  relationships  for 
the  entire  life  cycle  that  negates  or  severely  reduces  the  advantages  of  open  systems  designs.  A  trade-off  study 
should  be  made  to  determine  the  real  necessity  of  accepting  the  extensions  in  terms  of  performance,  cost- 
effectiveness,  and  supportability  factors. 

The  more  a  product  differs  from  standards  and  profiles,  the  more  a  customer  is  dependent  upon  the  OEM 
for  upgrades.  In  this  case,  the  uniqueness  of  an  initially-chosen  product  or  design  solution  becomes  a 
disadvantage.  An  OEM  may  be  more  expensive  than  anticipated,  go  out  of  business,  or  abandon  a  particular 
product  line  or  market  segment. 

2.4  Transitioning  to  an  Open  Systems  Environment 

Understanding  open  system  interface  standards  allows  the  evaluation  of  technical,  cost,  and  availability 
information  in  order  to  make  better  decisions  and  more  accurate  cost  projections  between  the  architectural 
alternatives.  Consequently,  the  challenge  is  to  assemble  an  effective  integrated  product  team  (IPT).  determine 
alternatives,  examine  and  make  comparisons,  and  contract  for  the  most  cost-effective  alternative.  The  use  of  open 
system  interface  standards  does  not  eliminate  the  need  for  meeting  programmatic  requirements  and  applicable 
laws.  However,  the  use  of  open  system  interface  standards  can  accelerate  the  actual  availability  of  products  and 
reduce  their  inherent  costs. 

2.4. 1  Availability  of  Information 

Information  needed  to  take  full  advantage  of  open  systems  falls  into  several  areas.  First,  the  entire 
acquisition  team  needs  an  overall  understanding  of  open  systems.  The  second  is  knowledge  of  the  individual 
standards  and  profiles  being  considered.  Third,  the  entire  acquisition  team  must  operate  from  a  common 
framework.  Information  about  individual  products  to  be  developed  or  bought  must  be  acquired  and  analyzed.  The 
acquisition  planning  IPT  should  know  the  following  about  a  product: 

•  Was  it  produced  specifically  for  commercial  or  government  markets? 

•  Is  it  accepted  in  those  markets? 

•  Have  upgrade  pathways  been  developed? 

•  Is  product  support  available? 

•  What  is  the  quality  of  the  product? 

•  Does  the  product  conform  to  the  standards  as  claimed  and  does  data  exist  to  provide  verification  ? 
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Individual  standards  and  technologies  information  is  available  from  various  sources.  A  number  of  trade 
associations,  standards  bodies  and  training  firms  have  developed  tutorials  on  specific  standards.  Detailed  product 
information  can  be  obtained  from  various  sources.  Industry  sources  include  databases  maintained  by  trade 
associations,  data  available  directly  from  OEMs  who  advertise  in  trade  publications  and  trade  publications 
themselves.  Related  DoD  Internet  links  include:  http://www.itsi.disa.mil  and  http://www.acq.osd.mil/osjtf 

Other  information  can  be  obtained  through  market  surveys.  Market  surveys  provide  acquisition  and 
supportability  information  for  NDI/commercial  item  products.  Market  surveys  can  also  be  used  to  evaluate 
commercial  OEM  interest  in  developing  new  open  system  interface  standards  based  products  to  meet  unique 
system  requirements,  e.g.,  MIL-STD-1397  type  B  (NTDS  FAST)  interface.  Even  in  a  purely  developmental 
situation,  the  interface  standard  (and  probable  silicon  chip  sets)  already  exists  and  is  ready  for  use. 

The  importance  of  conducting  market  surveys  and  trade-off  studies  prior  to  finalizing  a  performance 
specification  cannot  be  overemphasized.  How  to  apply  market  surveys  to  an  acquisition  is  discussed  throughout 
section  3  with  more  data  on  market  surveys  discussed  in  section  5.1.  Schedule  and  cost  comparisons  made  possible 
by  market  surveys  should  drive  the  selection  of  standards,  standard  profiles,  and  products.  This  task,  when  well 
done,  can  be  expected  to  provide  the  basis  for  much  more  accurate  and  defensible  planning,  programming,  and 
budgeting  system  (PPBS)  forecasts  than  was  the  case  in  traditional  stovepipe  acquisitions. 

2.4.2  Initial  Planning 

Planning  early  in  the  acquisition  life  cycle  can  turn  risks  into  opportunities.  In  an  open  system 
environment  this  involves: 

•  comparison  of  system  performance  requirements  to  the  inherent  capabilities  of  candidate  open  system 

interface  standard!  s), 

•  determining  the  relative  acceptance  of  the  candidate  standard! s)  in  commercial  markets, 

•  analyzing  and  comparing  alternative  products  that  implement  the  candidate  standard!  s)  for  suitability  in 

meeting  performance  requirements, 

•  anticipating  yearly  costs  for  each  alternative, 

•  predicting  initial  and  long-term  supportability  requirements  and  upgradeability,  and 

•  ensuring  no  deviation  from  open  standards. 

If  the  government  initially  purchases  a  commercial  item,  then  the  size  and  condition  of  that  product's 
market  becomes  a  driving  force  in  finding  and  selecting  competitive  products  for  upgrades  and  support.  If  the 
market  for  the  initial  product  is  large,  several  manufacturers  and  distributors  of  the  product  can  be  drawn  upon. 
Just  as  an  initial  purchase  leverages  the  commercial  marketplace,  bringing  its  cost  down,  the  same  investment  can 
be  leveraged  during  support  implementation  and  upgrading.  If  underlying  open  system  interface  standards  become 
obsolete,  then  the  military  will  need  to  upgrade  accordingly  unless  special  arrangements  are  made  (for  example, 
life-of-type  sparing). 

To  take  full  advantage  of  open  systems  benefits,  the  following  points  should  be  considered  when  planning 
system  acquisitions,  upgrades,  and  support. 


PLANNING  CONSIDERATIONS 

•S  Technologies,  standards,  profiles,  and  products  change  with  time  and  market  conditions. 

•S  Eliminate  proprietary  interfaces  unless  necessary  for  performance  considerations. 

•S  The  product  baseline  to  be  maintained  and  upgraded  must  be  based  on  standards,  profiles,  and  products  which 
implement  them. 

A  market  survey  is  a  critical  first  step  in  defining  what  can  be  accomplished,  the  costs,  and  when.  This 
includes  long  term  support  and  upgrade  planning,  life  cycle  cost  forecasts,  and  comparisons. 

S  Usage  of  open  products  to  provide  opportunities  to  compete  upgrades  and  support. _ 
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PLANNING  CONSIDERATIONS 

S  Market  surveys  must  be  kept  up-to-date  to  enable  support  and  upgrade.  Product  choices  must  be  based  on 
standards,  profiles,  and  supportability,  not  just  unique  performance  attributes. 

Conformance  of  products  to  selected  standards  and  profiles  is  critical.  Conformance  testing  provides  the 
foundation  for  successful  system  integration,  future  upgrades,  and  support. 

•S  Test  and  evaluation  of  products  and  product  upgrades  continues  throughout  the  system  life  cycle  and  must  be 
funded. 


2.4.3  Unique  Interfaces 

A  clear  upgrade  path  may  not  be  possible  if  unique  interfaces  are  used.  To  enable  smooth  upgrades  and 
product  support  after  an  initial  item  is  no  longer  available,  the  government  can  expect  to  pay  for  designing, 
integrating,  and  testing  upgraded  products.  Thus,  unique  interfaces  mean  cost  and  schedule  risks  that  must  be 
examined  in  the  early  phases  of  acquisition  planning. 

Proposed  deviations  from  commercially-accepted  open  system  interface  standards  need  to  be  evaluated  for 
the  relative  need  and  life  cycle  impact.  This  evaluation  should  be  made  during  the  planning  and  implementation 
of  initial  specifications.  One  method  encompasses  developing  a  list  of  interface  features  to  be  implemented  by  a 
prospective  developer  and  identifying  the  corresponding  performance  requirements.  The  program  office  must 
determine  if  the  developer  is  proposing  to  sell  unneeded  uniqueness  or  if  the  government  requirements  are  driving 
the  uniqueness.  If  the  government  requirements  are  driving  the  uniqueness,  then  the  program  office  should  re¬ 
evaluate  requirements  and  priorities.  A  second  scenario  might  be  when  a  proposed  design  calls  for  the  use  of  a 
product  but  the  contractor  claims  the  necessity  of  adding  features  to  a  product  fi.e.,  adding  a  daughter-board  to  an 
accepted  LRU).  How  this  product  change  affects  the  LRU  needs  to  be  carefully  evaluated  and  tested  prior  to 
acceptance. 

2.4.4  Risk  Management 

Risks,  and  the  management  thereof,  are  not  fundamentally  different  for  an  OSE;  however,  some  additional 
attention  needs  to  be  given  to  the  following  potential  risk  areas: 

•  Maturity  of  the  standard. 

•  Consensus  based  controlling  body  (avoid  proprietary  standards). 

•  Adequacy  for  military  use. 

•  Commercial  acceptance. 

•  Interoperability  of  products  via  the  same  stipulated  interface. 

•  Product  conformance  to  interface  standard  requirements. 

•  Use  of  proprietary  extensions  to  interface  standards. 

•  Product  interface  applications  modified  by  the  OEM  without  notification. 

•  Discontinuing  product  lines  without  warning. 

•  Changing  key  features  within  a  product  line  without  notification. 

•  Changing  parts  on  an  LRU  without  notification. 

•  Delivery  schedule  variances. 

The  risk  areas  must  be  mitigated  in  order  for  the  program  office  to  gain  the  advantages  of  open  systems.  Some 
considerations  to  reduce  risk  management  are  captured  in  the  following  table. 
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RISK  MANAGEMENT  CONSIDERATIONS 

•S  Are  risk  management,  mitigation,  and  abatement  efforts  addressed  by  the  program  office? 

•  Are  risk  abatement  modeling  and  simulation  tools  available? 

•  Has  the  staff  been  trained  to  employ  modeling  and  simulation  tools? 

•  Is  there  a  requirement  for  proof  of  performance  through  modeling  and  simulation  prior  to  down-selection  of 
product? 

•  Is  a  risk  management  plan  defined  and  roles  assigned? 

•S  Are  incentives  in  place  within  RFP/contract  for  risk  abatement  by  the  contractor? 

•S  Does  sufficient  test  data  exist  to  ensure  conformance  to  specified  standards  and  requirements? 

Are  the  selected  standards  and  profiles  changing  rapidly?  (Some  change  must  be  ongoing  or  the  standard  is 
obsolete.) 

S  Has  a  program  schedule  been  developed  and  adhered  to? 

•  Is  the  schedule  being  tracked? 

•  Have  critical  paths  been  identified? 

•  Are  critical  paths  tracked? 

•  Is  the  schedule  achievable  within  program  resources? 

•S  Are  vendor/subcontracting  schedules  part  of  the  planning/scheduling  effort? 

•S  Are  key  cost  drivers  identified  and  being  tracked? 

■>//  Has  an  ample  market  survey  and  conformance  test  effort  been  conducted  on  NDI/commercial  item  products  prior  to 
down-selection  and  integration? 

Have  prototyping  and  early  operational  testing  efforts  been  perfonned  before  finalizing  an  evolutionary  acquisition 
program? 

•S  Have  cost-effectiveness  studies  been  perfonned  and  issues  addressed  prior  to  prototyping? _ 


2.4.5  Quality  Assurance  (QA)/Management(QM) 

A  QA/QM  program  should  be  established  which  provides  a  methodology  oriented  toward  prevention,  detection, 
and  correction  of  defects.  If  the  contractor  has  an  acceptable  existing  QA  capability,  such  as  the  ISO  9000,  the  program 
should  incorporate  it.  In  an  OSE,  as  in  any  acquisition,  QA  should  be  planned  as  part  of  the  QM  process  and 
implemented  in  the  very  beginning.  Regardless  of  the  selected  QA  system,  the  program  office  needs  insight  into  the 
contractor  QA  efforts  so  that  contractual  compliance  is  monitored.  In  the  past,  QA  was  achieved  by  constant  testing  of  a 
product  throughout  the  product's  development  and  out  year  life  cycle.  Today’s  QA  environment  strives  to  establish  a 
process  (e.g.,  statistical  process  control)  for  monitoring  and  controlling  critical  processes.  The  QA  system  should:  have  a 
mechanism  for  feedback  of  field  product  performance;  implement  an  effective  root  cause  analysis  and  corrective  action 
system;  and.  include  a  continuous  process  improvement  program.  Key  questions  to  ask  are  provided  as  follows. 


QUALITY  ASSURANCE  CONSIDERATIONS 

•S  Has  the  integrator  and  associated  vendors/suppliers/subcontractor(s)  demonstrated  adherence  to  a  well  accepted  QA 
evaluation  process  (e.g.  SEI,  ISO  9000  series,  Baldridge  Award,  certification,  etc.)? 

Are  requirements  stipulated  in  the  SOW  that  require  QA  certification  or  compliance  standards  for  those  participating 
in  the  integration/development/production  of  the  system/equipment? 

•S  Has  conformance  testing  been  conducted  on  parts/components/LRUs  that  are  in  use  commercially  and  which  fulfill 
functional  requirement  and  open  system  interface  standards  applications? 

'ri  Is  environmental  stress  screening  part  of  the  integrator  and  vendor/supplier's  acceptance  process? _ 


2.4.6  Software  Engineering 
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Both  benefits  and  risks  result  from  the  transition  to  an  OSE,  especially  when  commercial  software  products  are 
incorporated  which  were  not  developed  for  the  program.  One  benefit  is  that  functional  and  performance  requirements 
may  be  satisfied  with  commercial  software.  This  spreads  costs  over  a  larger  marketplace,  reducing  life  cycle  costs  (LCC) 
to  DoD  users.  Each  program  office  should  track  commercial  products  it  uses  to  ensure  that  the  version  of  software 
integrated  into  the  design  is  not  modified  without  the  government's  knowledge.  However,  the  capability  to  track  version 
changes  is  dependent  on  configuration  management  policies  and  procedures. 

The  contract  specification  and  SOW  need  to  stipulate  performance  requirements  for  the  following: 

•  Software  adaptability. 

•  Portability. 

•  Scaleability. 

•  Documentation  of  extensions  to  standards/profiles. 

•  Test  documentation  (test  procedures  and  resultant  and  test  data)  to  facilitate  reuse. 

•  Functionality. 

•  Use  of  standard  data  definitions. 

•  Avoidance  of  proprietary  products. 

The  contract's  specification  must  state  how  these  attributes  will  be  verified  (inspected,  analyzed,  demonstrated  or  tested). 
These  same  requirements  apply  to  software  developed  specifically  for  the  program/system.  Again,  a  crucial  element  is 
deciding  which  specific  application  program  interfaces  will  be  used,  i.e.,  what  subset  commercially  based 
standards/profiles  will  form  the  interface  baseline. 

2.5  DoD  Initiatives  and  Requirements 

Open  systems  usage  is  part  of  a  larger  transition,  the  move  from  basing  defense  systems  on  DoD  unique 
interfaces  and  products  to  normative  use  of  commercial  standards  and  products,  modified  only  as  necessary  to  meet 
defense  needs.  It  is  not  the  intent  of  this  Guide  to  address  all  the  nuances  and  ramifications  of  this  change  but  to 
focuses  on  the  effects  of  open  standards  usage  (and  the  open  systems  environment  thus  created)  on  Navy 
acquisition  programs,  procedures,  and  costs.  Other  aspects  of  the  change  to  open  systems  are  addressed  throughout 
the  document. 

2.5.1  Business  Centered  Approach 

The  DoD  business  centered  approach  is  a  shift  of  strategy  and  tactics  toward  meeting  DoD  materiel  needs. 
Administrative  and  policy  aspects  of  the  business  centered  approach  include  cost  as  an  independent  variable 
(CAIV)  and  the  mandatory  use  of  commercially  based  open  system  standards  as  the  architectural  basis  for  system 
development  and  upgrade.  CAIV  is  the  methodology  to  acquire  and  operate  affordable  systems  by  setting 
aggressive,  achievable  cost  objectives  and  managing  achievement  of  these  objectives.  In  terms  of  an  OSE 
acquisition,  the  program  office  needs  to  establish  cost  objectives  to  balance  mission  need  with  projected  out-year 
resources  while  taking  into  account  anticipated  process  improvements  in  both  DOD  and  the  commercial 
marketplace.  In  place  of  the  traditional  “define  the  requirement  and  see  what  it  will  cost  to  meet  it”  approach,  we 
now  treat  cost  as  a  key  driver  in  the  choice  of  what  will  be  developed  or  purchased.  Unless  there  is  strong  reason 
to  do  otherwise,  capabilities  are  procured  to  the  extent  that  available  funding  allows,  not  necessarily  to  fully  meet  a 
defined  requirement.  Requirements  will  be  re-negotiated  in  light  of  availability  of  funds.  Expensive  programs, 
from  large  systems  to  small  upgrades,  may  be  canceled.  Within  this  constraint,  a  best  value  approach  is  to  be  used. 

The  business  centered  approach  places  DoD  acquisition  programs  in  a  new  and  different  position  relative 
to  industry.  In  the  past,  DoD  was  a  driving  force  and  sometimes  the  only  driving  force  in  development  of  high 
technology  products.  The  enormous  expansion  of  world  wide  commercial  markets  and  contraction  of  DoD  budgets 
has  made  industry  the  largest  investor  in  high  technology,  developing  leading  edge  products  to  meet  commercial 
market  needs.  This  change  has  vastly  increased  the  availability  of  a  wide  range  of  high  technology  products  which 
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can  meet  many  military  system  requirements.  In  fact,  there  are  often  choices  to  be  made  between  competing 
technologies  and  products,  each  of  which  may  partially  or  fully  meet  DoD  system  use  requirements. 

How  high  should  the  technology  be?  We  do  not  always  need  leading  edge  “state-of-the-art”  technologies 
and  products  to  meet  defense  needs.  Purchase  of  leading  edge  “state-of-the-art”  available  for  “laboratory  use  only” 
products  is  often  not  cost  effective.  There  are  real  advantages,  to  use  of  “state-of-the -practice”  products, 
particularly  for  fielded  systems. 

State-of-the-art  products  often  use  unique  interfaces,  which  may  and  often  do  change  prior  to  release  of 
production  run  quantities  of  products  for  commercial  market  applications.  Products  which  conform  to  open 
interface  standards  generally  have  well  thought  out,  well  understood,  and  well  documented  interfaces.  The 
stability  of  state-of-the-practice  product  interfaces  results  from  the  consensus  process  used  to  develop  and  test  them 
prior  to  release  as  approved  commercial  standards  and  use  by  industry. 

State-of-the-practice  items  generally  interface  via  open  interface  standards.  The  performance  of  these 
interfaces  is  a  known  factor.  Open  systems  interface  standards  are  readily  available  to  designers  who  in  turn  use 
the  standards  to  speed  design  and  achieve  predictable  performance  levels.  Because  other  designers  are  using  the 
same  interfaces,  integration  of  competing  or  complementary  products  is  easier.  The  achieved  level  of  commonality 
and  maturity  reduces  the  risks  (time  and  cost)  associated  with  product  integration,  making  system  design,  support, 
and  upgrade  easier  to  accomplish. 

Table  2-5  compares  the  business  aspects  of  these  alternatives. 


State-of-the-Art  Implies 

State-of-the-Practice  Implies 

Instability  of  interface  design,  rapidly  evolving  design 
changes 

Stable  interface  design 

Use  of  proprietary  interface(s)  to  capture  market  share 

Standardized  interfaces 

Questionable  performance  parameters 

Known  performance 

Difficulty  in  integration/interoperability 

Integration  easier  because  parameters  and  interface 
behaviors  are  known,  stable,  and  documented 

Lack  of  producibility,  low  production  rate 

Item  is  in  production 

Questionable  reliability 

Reliability  has  to  meet  commercial  expectations,  be 
high  enough  to  meet  cost,  warrantee,  and  reputation 
requirements 

Lack  of  technical  support  for  developers  and  integrators 

Technical  support  in  place  to  support  expansion  of 
product  market 

Lack  of  spare  parts  and  repair  support 

Spares  and  repair  available  -  routine  items. 

Lack  of  technical  data  to  enable  training  and  repair 

Technical  data  developed,  routinely  available 

Inability  to  order  large  quantities 

Production  is  underway,  needs  can  be  accommodated 

High  per  unit  cost 

Lower  per  unit  cost,  decreasing 

Delays  in  getting  the  product 

Product  routinely  available 

Table  2-5  State-of-the-Art  vs.  State-of-the-Practice 


2.5.2  Joint  Technical  Architecture  Overview 

With  the  growing  emphasis  on  Joint  operations,  numerous  efforts  at  the  DoD  level  are  underway  to  evolve 
legacy  systems  to  an  overarching  architecture  that  promotes  Joint  interoperability  and  acquisition  efficiency. 
Implementation  and  deployment  of  supporting  systems  will  be  realized  through  the  use  of  a  technical  infrastructure 
comprising  common  workstations,  core  capabilities  (applications)  across  functional  domains,  and  common  data 
elements  (i.e.,  data  structures).  This  commonality  will  be  predicated  upon  the  use  of  DoD  consensus  interface 
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standards  and  specifications,  and  the  availability  of  commercial  item  and  government  off-the-shelf  (GOTS) 
software  and  hardware  products  that  adhere  to  selected  common  standards  and  specifications. 

2.5 .2. 1  Architecture  B  ackground 

In  the  early  ‘90s,  the  Assistant  Secretary  of  Defense  for  Command.  Control,  Communications,  and 
Intelligence  (ASD  (C3I))  recognized  the  shortfalls  in  the  acquisition  of  systems  and  deployed  systems  capabilities. 
The  ASD  (C3I)  directed  the  Defense  Information  Systems  Agency  (DISA)  Center  For  Standards  (CFS)  to  perform 
the  role  of  executive  agent  for  all  DoD  information  technology  (IT)  standards  initiatives.  The  primary  objective  for 
the  CFS  has  been  to  identify  a  single  framework  to  promote  the  integration  of  DoD  information  systems.  This 
effort  resulted  in  the  development  of  the  DoD  Technical  Architecture  Framework  for  Information  Management 
(TAFIM)  which  establishes  the  direction  for  an  open  systems  environment  (OSE)  that  focuses  on  a  Standards- 
Based  Architecture  ( SB  A)  critical  to  achieving  interoperability  and  cross-functional  integration.  TAFIM  provides 
a  technical  reference  model  (TRM)  and  standards  selection  guidance  based  on  identified  Service  areas  (i.e., 
functional  areas)  and  DoD  consensus  standards  from  industry  and  DoD.  By  identifying  Service  areas,  the  TRM 
establishes  a  common  denominator  among  DoD  Commander-in-Chiefs  (CINC  (/Services/ Agencies  (C/S/A)  in 
which  to  select  and  specify  supporting  consensus  standards.  Implementation  of  TAFIM,  as  mandated  by  DoD 
5000. 2-R,  surfaced  ambiguities  with  respect  to  the  level  of  abstraction  in  which  to  apply  the  SBA  process;  and 
Joint  interoperability  given  multiple  and  competing  standards  for  an  identified  Service  area.  As  a  result,  a 
widespread  perception  formed  that  architectures  are  stovepiped,  piecemeal,  and  disjointed. 

In  October  1995,  the  C4ISR  Integration  Support  Activity  (CISA)  established  a  DoD-wide  C4ISR 
Integration  Task  Force  (ITF)  to  develop  coherent  integrated  C4ISR  architectures  framework.  In  June  1996,  the 
C4ISR  ITF  produced  the  CISA-0000- 10496,  C4ISR  Architecture  Framework  Version  1.0  document  which 
establishes  a  standard  set  of  rules,  products,  and  guidance  for  the  development  of  C4ISR  operational,  system,  and 
technical  architectures. 

In  November  1995,  the  ASD  (C4I)  issued  a  memorandum  to  DoD  C/S/As  to  establish  a  single,  unifying 
DoD  technical  architecture  that  will  become  binding  on  all  future  DoD  C4I  acquisitions  with  the  intent  to  broaden 
the  scope  to  later  address  weapon  systems  and  automated  information  systems  (AIS)  acquisitions.  This  effort 
resulted  in  the  development  of  the  DoD  Joint  Technical  Architecture  (JTA)  that  was  subsequently  mandated  in 
August  1996  for  DoD  wide  use  by  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology  (USD  (A&T)). 
The  purpose  of  the  DoD  JTA  is: 


•  To  provide  the  foundation  for  a  seamless  flow  of  information  and  interoperability  among  all  tactical, 
strategic,  and  sustaining  base  systems  that  produce,  use,  or  exchange  information  electronically. 

•  To  mandate  standards  and  guidelines  for  system  development  and  acquisition  which  will  significantly 
reduce  cost,  development  time,  and  fielding  time  for  improved  systems,  while  minimizing  the  impact  on 
program  performance  wherever  possible. 

•  To  influence  the  direction  of  the  information  industry's  standards -based  product  development  by  stating 
the  DoD's  direction  and  investment  so  that  information  industry's  development  can  be  more  readily 
leveraged  in  systems  within  DoD. 

•  To  communicate  DoD's  intent  to  use  open  systems  products  and  implementations  to  industry. 
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2. 5. 2. 2  Standards  Selection  Criteria 

The  standards  selection  criteria  used  in  developing  the  JTA  generally  focused  on  mandating  only  those 
items  critical  to  interoperability  that  were  based  primarily  on  commercial  open  system  technology,  were 
implementable,  and  had  strong  support  in  the  commercial  marketplace.  The  specific  guidance  given  in  selecting 
standards  was  that  standards  would  only  be  mandated  if  they  meet  all  of  the  following  criteria: 

•  Interoperability  and/or  business  case.  They  ensure  Joint  Service/ Agency  information  exchange  and 
support  Joint  (and  potentially  combined)  C4I  operations,  and/or  there  is  strong  economic  justifications 
that  the  absence  of  a  mandated  standard  will  result  in  duplicative  and  increased  life-cycle  costs. 

•  Maturity.  They  are  technically  mature  and  stable. 

•  Implementability.  They  are  technically  implementable. 

•  Public.  They  are  publicly  available  (e.g.,  open  system  standards). 

•  Consistent  with  authoritative  sources.  They  are  consistent  with  law,  regulation,  policy,  and  guidance 
documents.  (Note:  Recommendations  for  changes  to  these  authoritative  sources  should  be  made,  where 
necessary.) 

The  order  of  preference  used  to  select  the  standards  is  as  follows.:  Standards  that  are  commercially 
supported  in  the  marketplace  with  validated  implementations  available  in  multiple  vendors'  mainstream 
commercial  products  shall  take  precedence.  Publicly  held  standards  are  generally  preferred.  International  or 
national  industry  standards  are  preferred  over  military  or  other  government  standards. 

The  word  "Standards"  as  referred  to  in  the  JTA  is  a  generic  term  for  the  collection  of  documents  cited 
herein.  "Standards"  as  cited  in  the  JTA  may  include  commercial,  federal,  and  military  standards  and 
specifications,  and  various  other  kinds  of  documents  and  publications. 

The  JTA  replaces  the  standards  guidance  delineated  in  TAFIM  for  DoD  C4I  applicable  system 
acquisitions  and  is  recognized  as  a  living  document  that  will  evolve  with  time  as  technology  and  the  marketplace 
changes.  Application  and  requirement  specification  of  any  other  standards  (outside  of  those  identified  in  the  JTA) 
are  intended  to  be  additive,  complementary,  and  assumed  to  be  specific  and  unique  to  an  intraservice  mission  area 
requirement. 

2.5. 2. 3  Architectures 

It  is  important  to  understand  DoD's  big  picture  and  its  strategy  for  achieving  a  Joint  OSE.  DoD’s  big 
picture  is  to  define,  develop,  and  coordinate  Joint  target  architectures  (operational,  system,  and  technical)  that  are 
commensurate  with  projected  requirements  and  the  DoD  information  infrastructure  (DII).  Each  Service  is  a  cog  in 
the  picture  and  needs  to  ensure  that  the  systems  and  products  acquired  to  fit  into  that  picture  are  interoperable  and 
meet  the  performance  requirements  necessary  to  fulfill  the  mission  designated  for  the  product 

As  DoD  enters  into  a  new  century,  the  methods  of  supporting  operational  forces  will  be  significantly 
different  from  the  past.  Lessons  learned  have  driven  changes  to  acquisition  policies  and  the  realization  of  the  need 
to  revisit  our  method  of  supporting  the  Warrior.  Defining  and  providing  guidance  in  how  acquisition  and 
supportability  of  resources  to  meet  these  architectures  will  be  accomplished  and  defining  the  requirements  that  the 
program  office  must  fulfill  will  be  critical  to  success. 

The  DoD  JTA  development  is  predicated  upon  the  architecture  framework  concepts  and  guidance  depicted 
in  CISA-0000-104-963,  and  specifies  a  set  of  performance-based  commercial  information  processing,  transfer, 
content,  format  and  security  standards.  In  1995,  ASD  (C3I)  also  issued  a  task  which  established  the  C4ISR 
Integrated  Product  Team  now  called  the  Integration  Task  Force  (ITF).  The  ITF  assigned  the  Integrated 
Architectures  Panel  (IAP)  the  task  of  defining  the  architecture  process  and  to  address  the  architecture  planning 


3  C4ISR  Integrated  Architecture  Panel  ,  C4ISR  Architecture  Framework,  Version  1.0.  (4  June  1996). 
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and  engineering  concerns.  Part  of  the  IAP  effort  was  to  define  what  constitutes  an  architecture.  The  IAP  was  in 
agreement  with  the  IEEE  STD  610.12  definition  of  architecture  as  being:  the  structure  of  components,  their 
relationships,  and  the  principles  and  guidelines  governing  their  design  and  evolution  over  time.  To  further  clarify 
the  architecture  process  within  the  C4ISR  arena,  CISA-0000- 104-96  and  the  IAP  defined4  three  types  of  C4ISR 
architectures  as: 


•  Operational  Architecture.  Descriptions  of  the  tasks,  operational  elements,  and  information  flows  required 
to  accomplish  or  support  a  warfighting  functions. 

•  System  Architecture.  Descriptions,  including  graphics,  of  systems  and  interconnections  providing  for  or 
supporting  warfighting  functions. 

•  Technical  Architecture.  A  minimal  set  of  rules  governing  the  arrangement,  interaction,  and 
interdependence  or  the  parts  of  elements  whose  purpose  is  to  ensure  that  a  conformant  system  satisfies  a 
specified  set  of  requirements. 


The  two  references  document,  identify,  and  define  the  operational,  system,  and  technical  architectures 
(figure  2-3)  as  part  of  an  overall  architecture  framework. 


OPERATIONAL 


Technology  Forecast 
New  Technological 
Capabilities 


Processing 

Requirements 

lERs 


TECHNICAL 


Identifies  Warfiqhter  Needs 

Identifies  Standards  and  Conventions 

1/ 


SYSTEM 


Overlays  Capabilities  and 
Requirements  to  Identified  Standards 


lERs 

Command  Relationships 
Required  Capabilities  Matrix 
Node  Connectivity/Acticvity  Models 


Standards/Conventions/Profiles 
Technical  Criteria  Profile 
Technology  Forecast 


Note:  IER  (interface  exchange  requirements) 


Figure  2-3  Operational  -  System  -  Technical  Architecture  Relationships 


The  C4ISR  operational,  system,  and  technical  components  are  the  base  constructs  assumed  by  numerous 
and  ongoing  DoD  C4ISR  architecture  panels,  committees,  and  working  groups.  In  turn,  these  architectures  serve 
to  establish  a  set  of: 

•  normalized  activities  independent  of  command  structure, 

•  consensus  standards  among  C/S/As  with  the  objective  of  a  single  standard  for  a  functional  Service  area, 

•  TRMs,  and 

•  selected  standards  and  associated  standards  profiles  specific  to  each  Service  and  warfare  mission  area. 


4  C4ISR  Integration  Task  Force,  Integrated  Architectures  Panel  Final  Report,  September  1996. 
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The  DoD  OSE  approach  is  intended  to  promote  horizontal  and  vertical  interoperability,  use  of  open 
standards  based  systems,  and  faster  acquisition  cycle  times.  This  approach  encourages  functional  modularity  that 
can  be  interchanged  and  upgraded  with  improved  technologies  as  mission  and  budgets  dictate 

The  ability  to  leverage  commercial  products  to  meet  functional  requirements  within  a  DoD  system  became 
a  realistic  objective  and  has  led  DoD  to  expanding  the  evolutionary  acquisition  approach  into  including  the  use  of 
NDI/commercial  item  products  to  satisfy  architectural  requirements.  The  essential  engineering  issue  was  to  ensure 
the  interfaces  were  adequately  identified  and  profiled  to  ensure  interoperability 

The  currently  fielded  architecture  baseline(s)  address  the  existing  legacy  systems,  programs,  and  products. 
The  emerging  architectures  reflect  the  effort  to  migrate  from  these  legacy  systems  to  an  open  system  environment 
that  provides  for  Joint  interoperability  without  degradation  to  Service  unique  requirements.  This  means  that  the 
process  to  move  to  Joint  interoperability  must  first  achieve  approval  to  go  forward  in  the  acquisition  process  by 
meeting  Joint  requirements.  Program  offices  are  supposed  to  be  assessing  their  legacy  system  interfaces  (including 
data  protocol)  for  conformance  with  Joint  requirements.  As  legacy  interfaces  are  identified,  they  should  be 
compared  to  commercially  accepted  open  system  interface  standards  to  identify  opportunities  for  possible  migration 
to  commercial  products  in  meeting  Joint  interoperability  requirements.  As  operational  changes  occur  they  are 
being  assessed  for  how  they  impact  both  Joint  and  Service  unique  architectures. 

These  architectures  are  used  to  provide  the  user,  engineering,  and  acquisition  community  the  necessary 
perspectives  to  support  systems  acquisition  for  a  given  mission  area  and  functional  domain.  This  approach  by  DoD 
is  intended  to  promote  horizontal  and  vertical  interoperability,  open  standards  based  systems;  and  faster  cycle  times 
through  modularity  of  equipment’s  and  subordinate  functions  that  can  be  interchanged  and  upgraded  with 
improved  technologies  as  mission  and  budgets  dictate.  As  shown,  the  development  of  each  perspective 
(architecture)  results  in  information  (or  products)  that  serves  as  the  basis  for  the  other  two  architecture 
developments  and  definitions.  The  approach  is  a  continuum  paced  to  leverage  technology  advancements  for 
improved  capabilities  as  well  as  promote  interoperability,  scaleability,  and  reusability. 

2.5.3  TAFIM  Acquisition  Environment 

Traditional  DoD  acquisition  environments,  based  primarily  upon  proprietary  products  and  isolated  data 
processing  systems  have  resulted  in  a  costly,  poorly  integrated,  and  dosed  (rather  than  open)  infrastructure  in  most 
organizations.  The  Navy  recognizes  the  need  to  improve  the  use  of  existing  technology  while  simultaneously  developing 
new  acquisition  strategies  to  accommodate  declining  budgets,  significant  technological  advances,  and  changing  threats. 
Maintaining  current  and  leading  edge  technology  in  all  systems  demands  use  of  open  systems  to  promote  interoperability. 
The  thrust  of  DoD  acquisition  mandates  is  to  achieve  an  overarching  investment  balance  for  each  DoD  requirement. 

The  user,  system  engineering,  and  acquisition  communities  need  a  better  means  of  coping  with  this 
changing  environment  and  understanding  the  functions  being  supported  to  ensure  warfighter  needs  are  met  with 
open  systems.  Today,  the  program  office  needs  to  assume  a  SBA  planning  approach  if  they  are  comply  with  the 
specific  information  infrastructure  of  DoD,  Joint  needs,  and  satisfy  all  architectural  respective  requirements.  As  a 
means  to  assist  the  program  office,  DoD  has  changed  acquisition  policies  and  issued  a  revision  to  the  DoDI  5000.1 
and  DoD  5000. 2-R.  Included  in  the  changes  was  the  combining  of  the  policies  and  regulations  of  the  C4ISR. 
Combat  Weapons  Systems,  and  AIS  under  one  policy.  Part  of  this  policy  change  is  that  C4ISR  programs  consider 
the  Joint  requirements  in  their  acquisition.  During  the  development  of  the  mission  need  statement  (MNS)  and 
subsequently  the  operational  requirements  document  (ORD),  the  Service  and  multi-Service  requirements  must  be 
identified  and  stated. 
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For  those  programs  that  have  multi-Service  requirements,  the  MNS  and  ORD  have  to  be  approved  by  the 
authority  shown  in  table  2-6.  Concurrently,  the  Office  Assistant  Secretary  Of  Defense  (OASD)  has  mandated  the 
use  of  the  TAFIM  which  comprises  a  set  of  guidance  documents  for  developing  an  open  system,  including 
guidance  on  developing  an  SBA. 


Army 

Navy 

USMC 

Air  Force 

Requirements 

Developer 

TRADOC 

OPNAV  (N-8) 

Fleet  CINCs 

FMF/MCCDC 

Operating  Commands 

Service  HQ 

Lead 

Agency 

DCS  for  Operations  & 
Plans  (DCSOPS) 

OPNAV  (N-8) 
(Program  Sponsors) 

DCS  for  Requirements  & 
Programs 

DCS  for  Plans  & 
Operations  (HQAF/XO) 

Requirements  ACAT 

Validation  I/I  A 

Authority 

ACAT  I:  Joint  Requirements  Oversight  Council  (JROC) 

ACAT  IA:  OSD  Principal  Staff  Assistant  (PSA1/JROC 

ACAT 

n,m 

Chief  of  Staff 

Chief  of  Naval  Operations 

Commandant  of  Marine 
Corps 

Chief  of  Staff 

Requirements 

Documents 

Mission  Need  Statement  (MNS):  Milestone  0 

Operational  Requirements  Documents  (ORD):  Milestone  I,  II,  &  III 

Regulations 

AR  71-9 

AR  70-1 

OPNAVINST  5000.42D 
SECNAVINST  5000.2A 

MCO  3900.4D 

AFPD  10-6 

AFI  10-601 

Re:  Intermediate  Systems  Acquisition  Course  ACQ  201,  IS  AC  Volume  1,  Sept  96 


Table  2-6  Multi-Service  Approval  Authority 


The  TAFIM  comprises  eight  volumes  of  documentation  that  in  sum,  provides  a  TRM  and  standards 
selection  guidance  based  on  identified  Service  areas  and  DoD  consensus  standards  (Volume  7)  comprising  industry 
and  DoD  standards.  For  applicable  systems5 6,  the  JTA  replaces  the  standards  guidance  contained  in  the  TAFIM 
(Volume  if  By  providing  identified  Service  areas,  the  TRM  (Volume  2)  establishes  a  common  denominator 
among  DoD  C/S/ As  in  which  to  select  and  specify  supporting  consensus  standards  based  on  TAFIM  defined  work, 
application,  technology,  and  information  views.  Additionally  TAFIM  provides  a  systems  engineering  SBA 
planning  process  that  can  be  applied  to  any  level  of  abstraction  or  enterprise  in  determining  system  requirements. 

Provided  in  figure  2-4  is  the  seven  step  SBA  process  of  TAFIM.7  The  process  facilitates  and  promotes 
open  system  architectures  and  products  to  meet  performance  requirements.  The  SBA  process  offers  a  means  to 
structure  a  common  approach  for  the  definition  and  development  of  a  system  with  the  intent  to  enable 
interoperability  within  and  among  DoD  related  organizations.  Application  of  the  SBA  planning  process  provides  a 
systematic  approach  to  identify  acquisition  and  implementation  strategies  with  respect  to  requirements  definition 
and  standards  selection.  The  SBA  approach  supports  development  and  integration  of  individual  systems  and  Joint 
Mission  Areas  (JMA). 


5  For  a  definition  of  applicable  systems  refer  to  paragraph  4.3.4  of  DoD  5000.2-R. 

6  Refer  to  22  Aug  96  memo,  JTA  Implementation,  USD  (A  &  T)  and  ASD  (C3I). 

7  Refer  to  Internet:  http://www.itsi.disa.mil/cfs. 
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Figure  2-4  Seven  Step  SBA  Process  of  TAFIM 


The  SBA  provides  a  logical  engineering  processes  to  acquire  system(s)  in  a  timely,  cost-effective  manner, 
while  reducing  risk  and  ensuring  supportability. 
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3.0  ACQUISITION 

This  section  provides  a  process  for  achieving  a  cost  effective  and  best  value  open  system  acquisition 
approach  which  embodies  current  technology  over  an  entire  system’s  service  life.  Each  of  the  process  steps 
contained  in  this  section  are  not  fundamentally  new;  however,  accomplishing  the  objectives  of  an  open  system 
environment  (OSE)  as  described  in  section  2.0  affects  their  implementation. 

Pursuant  to  the  specification/standard  and  acquisition  reform  mandates,  the  focus  of  acquisitions  is  to  specify 
system  and  equipment  requirements  in  terms  of: 

•  end  item  performance  (vice  design  processes)-, 

•  hardware  and  software  interfaces  (interconnects  and  application  program  interfaces  (APIs))  predicated  upon 

consensus  standards  or  open  specifications;  and 

•  Joint  and  Naval  mission  areas  in  the  context  of  operational,  system,  and  technical  perspectives. 

Figure  3.1  provides  a  high  level  engineering  process  for  transitioning  from  a  mission  need  through  to  a 
selection/decision  process  for  open  system  acquisitions  that  correlate  directly  to  the  TAFIM’s  Standards-Based 
Architecture  (SBA)  concept. 


MISSION  REQUIREMENTS  DETERMINATION 


DEFINE  MISSION  PROFILE 


DEFINE  REQUIREMENTS  AND  STANDARDS 


S  Why  is  the  program  needed? 

S  Has  the  need  been  validated? 

S  What  specific  capabilities  are  necessary? 

S  Do  the  requirements  embrace  Joint  interoperability? 

S  When  are  the  necessary  capabilities  required  in  the  Fleet? 

S  How  much  will  the  program  cost? 

S  Is  the  program  affordable  and  fully  funded? 

^  Has  the  program’s  risk  been  assessed? 

^  Has  a  program  baseline  been  developed? 

S  Has  the  stability  of  the  design  and  operational  capability  of  the 
system  been  verified? 


S  Is  a  standards  approach  and  standards  profile  included  in  requirements  and 
acquisition  documents? 

S  What  is  the  acquisition  strategy  to  develop/produce/acquire  the  capability? 

S  Do  the  requirements  embrace  Joint  interoperability? 

^  Are  the  standards  referenced  in  the  profile  consistent  with  TAFIM,  and  DoD 
policy? 

S  Does  the  standards  profile  include  non-open  standards  or  proprietary 
technology(ies)? 

S  Are  there  any  major  voids  in  standards  required  by  this  system? 

S  If  a  selected  consensus  standard  is  negotiated  (deviates),  is  the  system  still 
OSE  compliant  and  does  the  system  still  conform  to  the  standard? 

S  Do  the  identified  standards  pose  interoperability  problems  when  compared  to 
profiles  of  other  systems  in  which  this  system  must  interoperate? 

S  Is  the  system  or  item  producible? 

S  Are  the  standards  widely  accepted  and  supported  by  multiple  vendors  (that  is, 
consensus  among  industry)? 

^  Do  products  currently  exist  to  support  the  standards  profile? 

S  Is  the  system  supportable? 


ACQUISITION  STRATEGIES  AND  OPTIONS 


DOCUMENTATION  AND  SOLICITATION 


CONTRACT  AND  DEPLOYMENT 


LIFE  CYCLE  MAINTENANCE  AND  UPGRADES 


Figure  3-1  Acquisition  Approach  and  Governing  Considerations 


Figure  3-1  correlates  the  process  flow  and  subordinate  processes  addressed  in  the  subsections  of  section  3. 
Each  of  these  subsections  address  a  segment  of  the  open  systems  process  and  provides  a  set  of  considerations.  It  is 
important  to  remember  that  in  an  OSE  acquisition  the  processes  do  not  always  have  to  be  performed  sequentially; 
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they  can  be  performed  concurrently  or  iteratively  as  appropriate.  When  risks  or  issues  arise,  actions  may  dictate  a 
revisit  to  a  particular  process  area  to  reaffirm  or  redefine  an  interface,  perform  a  market  survey  to  identify  new  or 
replacement  product(s)  or  technology  upgrades/replacement,  and  other  related  OSE  acquisition  considerations. 

Using  evolutionary  acquisition  and  acquisition  streamlining  initiatives  can  enable  the  acquisition  of  new  systems 
at  a  reasonable  cost  and  in  a  manner  that  responds  to  requirements  in  a  timely  fashion  (i.e.,  commensurate  with 
technological  advancements).  Proposed  systems  are  assessed  for  compatibility  and  interoperability  with  respect  to 
emerging  technologies,  product  availability,  and  application  to  legacy  environments.  Legacy  environments  are 
evaluated  for  Joint  and  unique  Service  adequacy  and  supportability,  and  the  potential  for  migrating  to  an  OSE. 
Figure  3-2  reflects  an  OSE  acquisition  process  and  identifies  the  corresponding  figures  in  section  3  which  portray 
the  process  described  in  each  block. 


START 


Figure  3-2  Iterative  Acquisition  Process 


DoD8  is  using  performance-based  contracts  to  leverage  commercial  markets  in  an  attempt  to  reduce 
acquisition  and  supportability  costs.  Acquisition  efforts  can  range  from  major  DoD  system  acquisitions  using  a  full 
developmental  approach  to  an  evolutionary  approach  which  employs  maximum  use  of  commercial  item  products  in 
the  design.  This  section  summarizes  the  processes  involved  in  acquiring  and  supporting  DoD  systems  using  OSE 
based  acquisition  concepts. 

3.1  Requirements  Definition 

A  program  office  changing  from  a  legacy  system  to  an  OSE  architecture  should  identify,  assess,  and 
document  all  hardware,  software,  and  internal  and  external  interface  requirements.  Interface  requirements  should 
reflect  actual  applications,  protocols,  etc.  necessary  for  system  performance.  Each  program  office  should  consider 


DoDI  5000.1  and  DoD  5000.2-R,  15  MAR  96. 
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establishing  an  integrated  product  teams  (IPT)  as  early  into  the  acquisition's  life  cycle  as  possible.  The  IPT  should 
have  DISA  participation  to  help  in  the  standard’s  selection  process  and  thereby  helping  to  expedite  the  Defense 


Naval  Warfare  Mission  Area  and  Required  Operational  Capability/Projected  Operational  Environment 
(ROC/POE)9  defines  the  interactions  between  the  Services  at  the  Joint  level.  "Requirements  for  new  or  modified 
systems  must  be  based  on  an  approved  need  that  addresses  their  use  in  Joint  operations  prior  to  requirement 
approval.”10  For  Joint  programs,  acquisition  and  requirements  documents  require  Joint  level  (i.e.,  J6)  review  and 
certification  to  ensure  interoperability  has  been  adequately  addressed. 

•  Requirements  documents  require  review  and  certification  by  J6  in  which  DISA  (supporting  Office 
Assistant  Secretary  Of  Defense  (OASD)  C3I)  is  a  key  participant  in  the  review  and  approval  of  the 
standards  approach,  standards  profile(s),  and  conformity  to  selected  standards. 

The  requirements  definition  process  shown  in  figure  3-3  may  be  used  to  assess  standards,  profiles,  and 
product  applicability. 


Figure  3-3  Process  Flow:  Requirements  Definition 


9  OPNAVINST  C3501.2H,  2  NOV  87. 

10  CJSCI  6212.01A. 
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The  result  of  the  requirements  definition  process  of  figure  3-3  is  the  approval  of  the  mission  need 
statement  (MNS),  operational  requirements  document  (ORD)  which  is  used  as  the  basis  for  developing  other 
programmatic  documentation.  The  initial  reference  model  and  functional  block  diagram  are  by-products  of  the 
MNS/ORD  approval  process.  The  following  ORD  and  MNS  considerations  should  be  addressed. 


MNS/QRD  CONSIDERATIONS 

•S  Is  the  MNS/ORD  consistent  with  respect  to  Joint  doctrine  (e.g..  Joint  Publication  6-0  and  6-2)? 

Is  the  MNS/ORD  consistent  with  current  concepts  for  Joint  or  Multinational  Forces  Operations  (e.g..  Joint 
Pub  6-0  series)? 

Is  the  proposed  system  consistent  with  the  DoD  migration  strategy? 

•S  Does  the  MNS/ORD  contain  a  statement  indicating  that  the  proposed  system  requires  the  following: 

•  Compatibility  with  existing  and  planned  C4I  systems  and  equipment? 

•  Interoperability  with  other  DoD  functionally  related  C4I  systems  and  equipment? 

•  Interoperability  with  Allied  nations’  functionally  related  C4I  systems  and  equipment? 

•S  Does  the  MNS/ORD  describe,  as  applicable,  key  boundary  conditions  related  to  C3I  interfaces  and 
standardization  or  interoperability  within  the  NATO  or  with  other  Allied  or  DoD  components? 

Are  any  new  strategic  or  nuclear  operations  interoperability  issues  raised  by  the  new  requirement? 

•S  Does  the  MNS/ORD  contain  a  standardization  approach? 

•S  Is  the  standardization  approach  consistent  with  the  Technical  Architecture  Framework  for  Information 
Management  (TAFIM)  and  DoD  Information  Technology  Standards  policy? 

S  If  any  standards  are  referenced  in  the  MNS/ORD.  are  they  consistent  with  the  TAFIM  and  DoD  Information 
Technology  Standards  policy? 

•S  Was  system  security  considered  as  a  constraint  that  could  adversely  impact  the  resolution  of  any  mission 
deficiencies? 

•S  Does  the  initial  reference  model  reflect  the  MNS/ORD? 

■S  Does  the  functional  block  diagram  reflect  the  MNS/ORD? 

Does  the  MNS/ORD  express  mission  need  to  the  extent  that  appropriate  roles  can  be  forecast  for  DoD 
strategic  and  tactical  common  user  communications  systems,  command  and  control  information  systems,  and 
National  Military  Command  System  (NMCS)  command  centers? 

•S  Is  the  Joint  Potential  Designator  identified? 

•S  Are  there  present  or  programmed  capabilities  that  could  fulfill  some  or  all  of  the  requirements  stated  in  the 
MNS/ORD? 

Are  interoperability  requirements  consistent  with  validated  architectures  for  the  function  being  supported? 

•S  Is  there  a  need  to  modify  existing  architectures,  or  to  develop  new  architectures  to  accommodate 

interoperability  requirements? _ 


3.2  Performance  Specification  Development 

An  OSE  implies  the  use  of  multiple  vendors  of  different  functional  products  whose  aggregate  products  are 
integrated  by  a  prime  contractor  or  system  integrator.  The  need  to  adequately  define  a  user's  requirements  is 
essential  to  an  OSE  acquisition.  The  performance  specification  is  the  primary  vehicle  to  define  program  functional 
and  performance  requirements.  A  key  part  of  the  OSE  process  is  the  transition  from  an  initial  set  of  basic 
operational  requirements  to  a  well  defined  performance  specification  that  defines  the  mandatory  interface 
requirements  necessary  to  achieve  an  OSE. 

Figure  3-4  provides  a  high  level  view  of  the  process  of  converting  initial  requirements  into  an  OSE 
performance -based  specification.  As  requirements  are  solidified,  the  data  (i.e.,  derived  requirements  generated 
from  the  MNS,  initial  market  survey  information,  and  an  initial  selection  of  interface  standards  and  associated 
profiles)  are  used  to  develop  a  draft  system  profile.  This  draft  profile  can  then  be  used  as  the  basis  for  development 
of  a  reference  architecture  model.  If  an  ORD  has  not  yet  been  developed  and  approved,  this  data  can  be  used  to 
solidify  the  ORD  requirements  and  to  develop  a  draft  performance  specification.  The  ORD  should  now  be 
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sufficiently  revised  to  describe  how  JTA  and  TAFIM  requirements  are  being  met.  The  draft  system  profile  should 
identify  a  set  of  standards/profiles  that  address  Service  unique  and  Joint  interface  requirements.  At  this  point,  a 
more  extensive  market  survey  should  be  performed  that  considers  system  profile,  reference  model,  performance 
specification(s),  and  especially  life  cycle  supportability  factors  including  related  cost,  schedule,  and  performance 
factors  of  end  items  under  consideration  for  integration. 


Figure  3-4  Process  Flow:  Performance  Specification  Development 


The  results  of  the  market  survey  (section  5.2)  are  used  to  make  programmatic  and  technical  acquisition 
decisions,  and  in  solidifying  design  and  acquisition  requirements.  In  order  to  refine  and  finalize  the  acquisition 
requirements,  the  market  survey  should: 

•  acquire  sufficient  data  to  fully  specify  the  functional  and  performance  needs  of  the  acquisition, 

•  identify  products  that  meet  the  standards  and  standard  profile  parameters  and  performance  requirements, 

•  evaluate  the  degree  to  which  products  meet  the  requirements,  and, 

•  determine  if  previous  conformance  and  interoperability  testing  results  are  available. 

Using  the  data  from  the  market  survey,  perform  a  trade-off  between  candidate  architectures  (groups  of  interface 
standards)  to  determine  if  and  what  development  effort  is  required  and  to  identify  the  initial  OSE  acquisition 
baseline. 


The  survey  data  and  final  performance  requirements  for  the  OSE  product  should  be  integrated  into 
acquisition  documentation  through  a  tiering  process.  Performance  specification(s)  and  mission  profile(s)  may  in 
fact  require  updating  to  reflect  the  market  survey  results.  How  well  the  derived  set  of  requirements,  standards 
profiles,  market  survey  analysis,  and  generation  of  the  OSE  performance  based  specification  are  performed  may 
directly  correlate  to  the  degree  of  risk,  cost-effectiveness  of  the  acquisition,  and  degree  of  achieving  an  OSE  within 
the  acquisition. 
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The  derived  set  of  requirements  can  be  viewed  as  a  tiering  of  standards  and  standard  profiles  as  shown  in 
figure  3-5.  As  lower  level  requirements  are  derived,  more  definitive  information  is  needed. 


System  Profile 


Figure  3-5  Tiering  of  Requirements 

A  tiering  process  is  usually  performed  in  two  steps.  First,  the  derived  set  of  requirements  are  cross-linked 
to  a  set  of  open  systems  interface  standards  which,  as  an  aggregate  listing  of  standards,  form  the  framework  for 
implementation  of  the  ORD  requirements.  Second,  each  identified  standard  currently  employed  within  a  legacy 
system  is  assessed  and  a  determination  made  regarding  which  of  the  mandatory  and  optional  requirements  (section 
2.2.1)  from  each  standard  is  necessary  for  interoperability.  The  tiering  process  results  in  the  initial  baseline  which 
includes  the  standards  listing  and  associated  standards  profiles.  The  interface  baseline  and  associated  cost  analysis 
data  can  be  used  to  refine  the  performance  requirements  in  the  higher  level  documentation  (ORD,  etc.). 

As  part  of  the  market  survey,  perform  an  assessment  of  candidate  architectures  and  how  well  their 
associated  products  fulfill  an  identified  function  or  set  of  functions.  Then  assign  a  set  of  cost  and  technical 
parameters  to  the  candidate  architectures  so  that  when  data  for  these  parameters  is  accumulated,  a  matrix  can  be 
developed  and  a  value  and/or  risk  factor  can  be  assigned  to  each  parameter  as  shown  in  figure  3-6.  Entries  in 
figure  3-6  provide  the  IPT  with  the  types  of  capabilities  and  risks  (confidence  factors)  associated  with  the 
acquisition.  Figure  3-6  is  an  example  format  for  a  summary  comparison  matrix.  The  matrix  must  be  tailored  for 
each  individual  program's  unique  requirements.  For  example,  if  products  are  being  evaluated  against  the  initial 
standards  and  standard  profiles,  then  an  assessment  can  be  made  of  how  well  the  products  match  the  requirements. 
Test  results  can  be  reviewed  to  determine  the  product’s  capacity  to  conform  and  interoperate  within  program 
needs.  If  there  is  no  product  match  to  standards  and  standard  profiles,  then  some  combination  of  existing 
NDI/commercial  item  or  an  unique  product  development  is  required. 
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Figure  3-6  Summary  Comparison  Matrix 

In  addition,  sufficient  data  should  now  be  available  to  the  IPT  to  consider/identify  options 
(Implementation  and  Migration).  The  profiles  necessary  for  decisions  (functional,  environmental,  and 
supportability  profiles)  should  be  fairly  sound  with  only  some  review  necessary  depending  on  the  final  selection  of 
an  acquisition  strategy.  For  instance,  the  outcome  of  the  market  survey  may  require  the  supportability  profile  to  be 
revisited  prior  to  any  decision  to  field  the  system/product  by  the  integration  of  Non-Development  Item  (NDI)  and 
commercial  item  products. 

3.2.1  Life-Cycle  Cost  Comparisons  of  Candidate  Architectures 

Traditionally,  life-cycle  support  of  a  system  was  planned  during  engineering  and  manufacturing 
development  (E&MD)  or  production.  By  the  beginning  of  E&MD,  85  percent  of  life  cycle  support  costs  had  been 
determined  by  the  system  definition.  In  order  to  mitigate  this  cost  risk,  supportability  trade-off  studies  need  to  be 
performed  prior  to  the  selection  of  standards  and  products.  A  life  cycle  cost  forecast  (i.e.,  costs  for  development, 
facilities,  and  out-year  tasking  to  the  ISEA/CFA,  SSA,  T&E,  and  other  program  support  entities)  should  be 
developed  for  each  candidate  architecture.  These  costs  are  driven  by  the  rate  of  change  of  NDI/commercial  item 
products. 

Even  in  a  nominally  pure  developmental  situation,  using  open  system  interface  standards  as  the  basis  for 
system  design  promotes  the  use  of  NDI/commercial  items,  though  at  a  lower  design  level.  Interface  related 
hardware  products  include  chip  sets,  connectors,  fiber  optic  cable,  etc.  Interface  related  software  products  include 
operating  systems,  development  environments,  programming  languages,  compilers,  debugging  tools,  etc.  Each  of 
these  products  has  existing  supportability  parameters,  ranging  from  repair  to  technical  consultation,  upgrades,  and 
licenses.  These  parameters  must  be  analyzed  and  translated  into  probable  program  planning,  including  schedule, 
and  cost.  Life  cycle  cost  and  scheduling  for  each  candidate  architecture  should  be  factored  into  the  IPT's  decision 
as  to  which  architecture  provides  the  best  cost/performance  tradeoff  value11. 

Most  open  system  interface  standards  based  situations  will  involve  mixtures  of  developmental  and  non- 
developmental  products.  To  take  advantage  of  available  open  system  and  product  information,  business 
management  and  supportability  personnel  must  be  involved  in  the  IPT  process  during  the  early  phases  of  the 
program  and  the  market  survey  process.  The  challenge  is  to  integrate  developmental  and  NDI  support  elements, 
not  only  at  the  product  level  but  also  at  the  system  level.  Associated  with  this  is  a  schedule  challenge. 


11  DoD  5000.2-R,  15  MAR  96,  section  3.3.3. 
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Supportability  products  (e.g.,  technical  manuals,  technical  support  services)  must  be  planned  and  made  available 
for  system  design,  training  development,  T&E,  installation  and  checkout,  and  fleet  introduction. 

3.2.2  Reference  Models 


Reference  models  provide  the  framework  to  build  a  common  understanding  and  agreement  for  a  system. 
A  well-defined  TRM  is  the  building  block  that  facilitates  an  abstract  framework  to  generate  a  common 
understanding  from  both  a  horizontal  and  vertical  system  architectural  viewpoint.  This  promotes  a  consensus 
among  system  participants  as  to  what  constitutes  the  technical  architecture  and  identifies  any  issues  requiring 
resolution.  The  TRM  provides  the  basis  for  building  a  target  system  architecture  as  illustrated  in  figure  3-7. 
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Figure  3-7  OSE  Architecture  Model  Compared  to  Warfare  Related  TRM 


Based  on  the  target  system  architecture  defined,  determine  which  standards  and  standard  profiles  are 
needed.  Select  open  system  interface  standards  not  only  on  the  basis  of  openness,  maturity,  and  satisfaction  of 
performance  requirements  but  also  the  capability  to  facilitate  future  technology  insertion.  Other  considerations  in 
the  selection  process  include  scaleability,  portability,  and  interoperability.  Open  system  interface  standards  usually 
have  profiles  of  mandatory,  optional,  configurable,  and  extension  features.  Selected  configurable  features  should 
not  be  vendor  specific  and  vendor  extensions  should  be  avoided  when  selecting  standards’  requirements  and 
features.  This  also  applies  during  product  selection;  otherwise,  the  benefits  and  openness  of  the  design  are  greatly 
reduced. 


An  example  of  the  relationship  between  standards  and  a  TRM  is  provided  as  figure  3-8.  At  this  point 
there  should  be  sufficient  data  to  determine  the  effort  needed  to  redefine  interface  requirements  to  address 
interoperability  requirements.  The  ORD  should  be  revised  to  determine  if  the  defined  TRM  results  need  to  be 
included  in  the  ORD. 
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Figure  3-8  Sample  TRM  of  an  OSE  Operational  System 


3.2.3  Integrated  Product  and  Process  Development  (IPPD)/Integrated  Product  Teams  (IPT) 

IPPD  is  a  systematic  approach  used  by  the  program  office  for  early  integration  and  application  of  all 
disciplines  for  system  acquisition  and  life -cycle  support.  A  key  proponent  of  the  IPPD  is  the  use  of  IPTs. 

The  IPT  concept  allows  the  program  office  to  use  functional  experts  to  resolve  technical  and 
programmatic  issues;12  therefore,  the  IPTs  must  be  comprised  of  personnel  with  the  appropriate  expertise  and  skill 
sets  for  the  program.  The  IPT  should  consist  of  a  cross-populated  set  of  technical.  Program  management,  and 
contracts  personnel  such  as:  ISEA/CFA,  SSA.  design  agent(s),  integrator,  possibly  industry/contractor,  and  other 
activities  representatives  as  necessary  to  ensure  an  adequate  and  cost  effective  OSE  product  is  procured  and 
supported.  In  addition  to  the  normal  IPT  functions,  the  following  OSE  IPT  functions  also  apply: 

•  Defining  the  interface  standards  profiles  of  legacy  system/products. 

•  Comparing  the  legacy  profile  to  the  Joint  profile  as  needed. 

•  Generating  an  initial  list  of  Navy/Joint  open  systems  interface  standards. 

•  Coordinating  standard/ s)  profile  w/  DISA  center  for  standards  (CFS)  (TAFIM  compliance 
considerations). 

•  Participating  in  an  OSE  market  survey  analysis. 

•  For  example,  the  contracting  officer  (a  member  of  the  IPT)  needs  to  be  knowledgeable  about  the  nature 
and  impact  of  open  systems.  This  knowledge  must  encompass  the  legal  aspects  of  copyrights,  contracts, 
and  performance  specifications  (wording  and  tiering).  The  contracting  officer  must  be  available  and 
prepared  to  work  closely  with  the  program  office  staff,  ISEA/CFA,  SSA,  design  agent,  integrator,  and 
other  activities.  The  contracting  officer  must  be  positioned  to  quickly  place  and  modify  contracts  as 
driven  by  changes  to  standards,  product,  product  support,  and  service  availability.  These  changes  may 
result  in  the  need  to  procure  new  item(s)  or  life-of-type  buys.  This  implies  additional  testing  and 


12  DoD  5000.2-R,  15  MAR  96,  section  1.6. 
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coordination  between  the  testing,  integration,  and  support  elements.  Contracting  actions  can  be  expected 
to  be  more  time  critical  than  in  conventional  development  acquisitions. 

The  following  represents  the  type  of  questions  the  IPT  should  address  as  part  of  the  functional  design 
considerations. 


FUNCTIONAL  DESIGN  CONSIDERATIONS 

•S  Was  the  analysis  successful  in  defining  any  lower  level  functional  and  performance  requirements  including 
functional  interface  and  architectures? 

•S  Have  any  interface  impacts  been  identified  between  the  legacy  set  of  interface  standards  and  those  needed  for 
potential  change  implementation? 

•S  Have  commercial  specifications/standards  changes  been  tracked  and  analyzed  for  impact? 

•S  Has  the  standards  listing  been  updated/coordinated  at  the  Joint  level  to  reflect  new  requirements? 

•S  Have  trade-off  studies  been  performed  on  the  current  or  recommended  interface  standards? 

•S  Has  a  configuration  management  (CM)  plan  that  addresses  standards  based,  NDI  and  commercial  item  applications 
been  developed  and  implemented? 

•S  Has  an  initial  logistics  and  supportability  process  (with  its  associated  procedures)  been  defined? _ 


3.3  Acquisition  Strategies  and  Options 

The  processes  performed  to  this  point  should  provide  the  program  office  with  a  good  understanding  of  the 
requirements;  availability  of  products  to  fulfill  these  requirements;  and  alternatives  for  reducing  risk.  As 
performance  requirements  are  finalized  to  reflect  the  initial  market  survey  and  trade-off  studies,  the  accumulated 
information  (including  any  identified  products)  may  be  used  to  develop  alternative  acquisition  strategies.  Using 
the  data  available,  the  IPT  can  determine  whether  to  proceed  with  a  full  NDI/commercial  item  procurement;  a 
mixture  of  developmental  and  NDI/commercial  item;  or  a  full  developmental  approach.  Realistic  cost- 
effectiveness  data  for  each  alternative  acquisition  approach  should  be  postulated. 

Figure  3-9  represents  a  simplified  flow  process  for  identifying  the  strategy  and  defining  the  options 
available  to  the  IPT. 


Figure  3-9  Process  Flow:  Identifying  Acquisition  Strategies 
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3.3.1  Acquisition  Scenario  Discussions 

Using  the  best  information  available,  the  program  office  can  select  the  initial  acquisition  strategy.  The 
acquisition  strategy  may  be  refined  as  the  requirements  and  specifications  are  developed  for  the  RFP.  Several 
different  scenarios  for  acquiring  products  can  be  undertaken  to  achieve  the  same  results.  Several  scenarios  and 
three  strategies  that  can  be  applied  are  listed  below: 

•  Acquisition  Scenarios 

•  Government  selects  the  standards  and  products  then  performs  the  integration  effort. 

•  Government  selects  the  standards,  the  contractor  designs  or  selects  products  and  integrates  the 
system. 

•  Government  and  contractors  share  the  responsibility  of  selecting  the  standards  and  integrating  the 
products. 

•  Contractor  selects  the  standards  and  products  then  performs  the  integration  effort. 

•  Acquisition  Strategies 

•  Purely  developmental 

•  Mixed  developmental  and  NDI/commercial  item 

•  Pure  NDI/commercial  item  integration 

3. 3. 1.1  Government  Selects  the  Standards  and  Products  and  Performs  the  Integration  Effort 

Under  this  scenario,  the  government,  based  on  internally-held  knowledge,  makes  and  implements 
acquisition  decisions.  This  approach  depends  upon  the  collective  knowledge  in  System  Command  (SYSCOM) 
headquarters,  warfare  centers,  or  Navy  laboratories.  The  government  functions  as  design  agent,  systems  integrator, 
producer,  and  installer.  The  government  selects  and  specifies  open  system  interface  standards,  profiles,  and  open 
system  interface  standards  based  commercial  item  products.  The  government  then  buys  and  integrates  the  products 
into  the  overall  system.  Making  such  standards  and  commercial  item  selections  requires  the  government  to  make  a 
strong,  up-front  investment  (time  and  money)  in  personnel,  training,  and  facilities. 

This  scenario  presumes  that  government  personnel  have  the  necessary  knowledge  to  perform  and  interpret 
market  surveys  to  accurately  define  and  implement  a  program.  Initial  market  survey  and  analysis  work  are 
conducted  during  Concept  Development  and  Demonstration/Validation.  Additional  market  surveys  are  required  as 
upgrades  and  product  obsolescence  are  encountered.  Associated  with  the  market  survey  are  trade-off  studies 
required  for  reuse,  reverse  engineering,  and  re-engineering,  process  improvement,  and  creative 
procurement/acquisition  strategies  as  required  to  take  maximum  advantage  of  the  open  system  environment.  A 
team  of  engineers,  logisticians,  acquisition  planning  personnel,  and  cost  personnel  is  required  to  develop,  conduct, 
and  interpret  trade-off  studies. 

The  government  must  be  postured  to  respond  to  standards  updates,  new  standards,  and  the  resulting 
impacts  on  purchased  commercial  item  products.  When  changes  to  a  standard  directly  affects  products  being 
procured,  that  change  must  be  addressed.  Impacts  resulting  from  standards  changes  may  affect  software 
portability,  software  reuse,  and  faster  technology  insertion.  System  definition  likely  will  take  more  time  due  to  the 
need  to  perform  trade  studies,  gather  data,  and  so  forth.  System  design  may  be  faster  due  to  commercial  item 
availability  and  the  existence  of  mature  interface  definitions. 

3.3. 1.2  Government  Selects  Standards,  Contractor  Designs  or  Selects  Products  and  Integrates  the  System 

Under  this  scenario,  as  in  Section  3. 3. 1.1,  the  government  makes  the  decisions  regarding  standards  and 
profiles  to  be  used  in  developing  and  integrating  products.  The  contractor’s  task  is  to  design  or  purchase  open 
system  interface  standards  based  products,  then  integrate  them  into  a  functional  system. 

In  this  scenario  it  is  important  to  ensure  that  a  contractor's  stated  choices  of  interface  standards  and 
profiles  are  accurately  implemented  in  the  products  selected  for  purchase  or  development.  If  the  interface 
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standards  are  not  strictly  adhered  to,  resultant  products  will  not  function  together  effectively  or  may  be 
incompatible  with  upgrades  built  to  the  same  standards.  Contract  data  requirements  lists  (CDRLs)  to  provide 
interface  information  and  product  testing  are  important  for  risk  reduction.  The  government  needs  to  be  able  to 
accept  or  redirect  the  contractor' s  efforts  and  to  insure  that  they  do  indeed  result  in  a  system  that  conforms  to  the 
interface  as  well  as  other  performance  requirements. 

3.3. 1.3  Government  and  Contractors  Share  Responsibility  of  Selecting  Standards  and  Integrating  Products 

This  scenario  represents  the  middle  ground  followed  by  most  acquisitions.  To  obtain  the  benefits  of  the 
best  resources  available  when  planning  an  acquisition,  the  government  invites  industry  to  participate  in  the 
interface  and  profile  decision-making  process,  and  the  selection  of  compliant  products.  This  may  be  accomplished 
through  use  of  Joint  working  groups,  funding  of  studies,  and  providing  draft  request  for  purchases  (RFPs)  for 
comment.  This  scenario  requires  the  contractor  to  be  familiar  with  legacy  systems  (for  upgrades),  the 
requirements,  the  technologies  applicable  to  other  aspects  of  the  platform  or  system,  and  industry  trends.  The  goal 
of  this  scenario  is  to  apply  this  knowledge  during  the  standards  selection  process,  while  considering  the  long-range 
plans  for  interface  and  product-level  standardization  within  or  across  program  and  Service  lines. 

In  this  scenario  it  is  important  to  ensure  that  a  contractor's  stated  choices  of  interface  standards,  and 
profiles  are  accurately  implemented  in  the  products  selected  for  purchase  or  development.  If  the  interface 
standards  are  not  strictly  adhered  to,  resultant  products  will  not  function  together  effectively  or  may  be 
incompatible  with  upgrades  built  to  the  same  standards.  CDRLs  to  provide  interface  information,  and  product 
testing  are  important  for  risk  reduction. 

To  define  and  use  evaluation  criteria  and  conduct  trade-off  studies  for  selection  of  interface  standards, 
technologies,  and  products,  the  government/contractor  team  must  understand  what  are  the  applicable  open  system 
interface  standards  choices,  the  technologies,  products,  and  relative  costs,  and  what  can,  and  cannot  be 
accomplished.  This  knowledge  base  may  not  be  easy  to  assemble.  The  range  of  risks  and  choices  is  bounded  by 
the  technologies,  the  standards  and  product  maturity,  and  by  market  factors. 

This  scenario  might  occur  if  the  goal  of  a  program  is  to  design  an  entire  aircraft.  Overall  performance 
requirements  (e.g.,  speed,  range,  payload,  accuracy  of  navigation,  crew  size,  and  information  to  be  provided  to  that 
crew)  are  set  by  government.  The  method  of  achieving  those  performance  requirements,  though,  is  not  set.  In 
some  cases,  it  may  be  sensible  to  have  the  contractor,  not  the  government,  decide  which  open  system  interface 
standards,  and  profiles  apply  in  meeting  these  performance  objectives.  Legacy  items,  and  the  standards,  and 
profiles  to  which  they  were  designed  obviously  must  be  accommodated.  Government  participation  is  critical 
during  the  entire  life  cycle. 

Interface  and  commercial  item  decisions  can  be  made  prior  to  the  final  RFP  release,  concurrent  with 
contract  award,  or  after  contract  award.  Some  considerations  for  the  selected  options  are: 

•  Prior  to  issuing  the  final  RFP  -  Under  this  option,  many  different  types  of  participants  may  become 
involved  in  the  process  of  choosing  the  final  interface,  and  commercial  item  products  to  be  solicited. 
Contracts  may  be  issued  for  support  of  such  studies  by  one  or  more  contractors  in  a  consortia  based 
working  group  situation.  Alternately,  a  single  contractor,  acting  as  a  consultant,  may  perform  such 
studies.  Academia  such  as  university  laboratories  may  participate  as  well.  Navy  warfare  centers,  and 
cost,  and  supportability  analysts  also  should  be  part  of  the  team.  Studies  performed  by  these  participants 
may  be  used  to  develop  a  solid,  and  complete  performance  specification,  including  interface  definitions, 
for  inclusion  in  the  final  RFP.  The  greater  the  quality  of  participants,  the  better  the  quality  of  the  ultimate 
specification.  Award  protests  also  should  be  diminished  due  to  such  broad  participation  in  interface,  and 
commercial  item  selection  from  the  outset. 
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•  Make  selection  part  of  proposal/award  -  In  this  option  the  Navy  team  must  have  the  information  to 
evaluate  the  proposals  and  make  the  appropriate  product  selections.  This  introduces  risk  by  requiring  the 
government  to  be  knowledgeable  in  open  system  interface  standards  and  associated  products. 

•  After  contract  award  -  This  may  be  an  appropriate  option  when  major  system  elements  are  not  yet  fully 
defined.  However,  this  option  allows  for  fewer  inputs  to  the  interface  and  commercial  item  choices,  and 
thereby,  greater  susceptibility  for  contract  modifications. 

3.3. 1.4  Contractor  Selects  Standards,  and  Products,  and  Performs  the  Integration  Effort 

Under  this  scenario,  the  contractor  performs  the  analysis  work  necessary  for  standard,  and  product 
selection,  and  for  system  integration.  A  system  design  review  (SDR)  should  be  conducted  to  allow  the  contractor 
to  present  and  prove  the  appropriateness  of  the  standards  and  commercial  item  selected.  Profile 
selection/development  and  production  selection/development  then  become  the  subject  of  the  preliminary  design 
review  (PDR)  and  the  critical  design  review  (CDR). 

When  the  contractor  is  responsible  for  open  system  interface  standard  and  product  selection,  the  following 
SOW  language  may  be  employed.  The  language  presented  supports  the  market  analysis  that  should  provide  the 
government  with  needed  information  for  review  and  approval. 

The  contractor  shall  perform  trade  studies  and  market  surveys  comparing  open 
system  interface  standards  and  products  being  considered  for  use.  The  market  survey  shall 
include  a  discussion  of  the  availability  of  commercial  item  products  and  their  ability  to  meet 
requirements  and  supportability  plans,  including  spares  procurement  and  lifetime  product 
repair.  The  market  survey  also  shall  provide  advance  planning  for  replacement  of  obsolete 
products.  For  hardware,  the  market  survey  shall  include  multiple  product  sources  at  the 
functional  lowest  replaceable  unit  (LRU)  level  and  identify  whether  products  are  pin-for- 
pin  interchangeable  (i.e.,  require  software  changes  for  use  of  the  replacement  product).  For 
software,  the  market  survey  shall  include  products  that  are  open  system  interface  standards 
compliant  when  resident  on  hardware  products  being  evaluated  and  selected.  The 
contractor  shall  justify  ultimate  product/design  selection  and  provide  supporting 
documentation  as  a  technical  report  and  presentation  to  the  government  for  approval  prior 
to  proceeding  with  design.  In  the  event  that  the  interfaces  and/or  products  selected  do  not 
meet  system  requirements,  the  contractor  shall  be  prepared  to  provide  technical  and 
programmatic  data  necessary  for  obtaining  any  required  waivers.  The  contractor  shall 
perform  or  cause  to  be  performed,  requisite  testing  to  ensure  conformance  of  developmental 
and  commercial  item  products  to  selected  standards  and  profiles. 

Without  an  understanding  of  the  implications  of  each  of  the  standards,  profiles,  and  products  considered 
and  not  considered  by  the  contractor,  the  government’s  review  of  a  contractor’s  selections  cannot  accurately 
address  analysis,  cost  effectiveness,  or  appropriateness.  This  scenario  requires  the  contractor  to  be  familiar  with 
legacy  systems  (for  upgrades),  the  requirements,  and  technologies  applicable  to  other  aspects  of  the  platform  or 
system,  and  more  importantly,  industry  trends.  Nonetheless,  when  the  government  team  does  not  have  access  to 
requisite  information  and  skills,  this  may  be  a  good  strategy. 

In  this  scenario  it  is  important  to  ensure  that  a  contractor's  stated  choices  of  open  system  interface 
standards  and  profiles  are  accurately  implemented  in  the  products  selected  for  purchase  or  development.  If 
interfaces  are  not  strictly  adhered  to,  resultant  products  will  function  together  ineffectively  or  may  be  incompatible 
with  upgrades  built  to  the  same  standards.  CDRLs  to  provide  interface  information  and  product  testing  are 
important  for  risk  reduction. 

3.3.2  Acquisition  Strategy  Considerations 
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Risk  can  be  mitigated  by  generating  a  draft  RFP  which  includes  a  draft  statement  of  work  (SOW)  and 
performance  specification  which  could  be  forwarded  to  interested  industry  participants  for  comments.  Industry 
would  be  solicited  to  provide  inputs/suggestions  on  the  RFP  to  the  government;  thereby,  enabling  the  program 
office  to  identify  standards,  standards  profile  considerations,  and  associated  products  capable  of  meeting  the 
requirements  which  were  not  identified  during  the  market  survey.  Additionally,  industry  may  identify  problems 
among  commercial  products  currently  being  fielded  or  with  previously  stipulated  requirements  and  technology  that 
had  not  been  known  or  considered.  There  is  a  drawback  to  this  approach.  A  manufacturer  may  mistakenly 
overstate  the  capabilities  of  their  product,  a  product  they  recommended  for  integration,  or  understated  another 
manufacturer’ s  product. 

The  following  considerations  are  typical  acquisition  strategy  questions  a  program  office  should  take  into 
account  when  preparing  to  define  and  document  an  OSE  acquisition  program. 


ACQUISITION  STRATEGY  CONSIDERATIONS 

•S  Are  any  NDI  products  available  to  meet  the  needs  of  the  program? 

•S  Do  sufficient  commercial  item  products  exist  to  proceed  with  full  integration? 

•S  Do  commercial  item  production  schedules  meet  program  needs? 

How  are  the  commercial  items  supported  and  do  they  meet  program  needs? 

•S  Are  there  any  second  source/multiple  vendor  capabilities? 

Have  sparing  requirements  been  identified  and  are  they  ample  to  meet  program  needs? 

Has  a  logistics  analysis  been  performed  including  cost-effectiveness  study? 

Have  training  requirements  been  addressed  and  solidified? 

S  Have  software  support  requirements  been  determined? 

Has  a  risk  assessment  been  performed? 

Have  documentation  needs  been  established  (minimum  data  to  manage  and  support  program)? 

•S  Will  a  level  of  repair  analysis  (LORA)  be  accomplished  prior  to  RFP  or  will  the  contractor  be  required  to 
perform  it? 

Have  the  T&E  requirements  been  defined? 

•S  Does  sufficient  conformance  and  interoperability  test  information  exist? 

Have  the  commercial  item  product(s)  life  cycle  been  determined? 

•  How  often  is  the  item  upgraded? 

•  Is  it  part  of  a  product  line? 

•S  Does  the  funding  profile  meet  program  needs  and  does  it  consider  the  potential  for  evolutionary  acquisition 
upgrades?  (e.g.,  every  3-5  years?) 

•S  Does  sufficient  commercial  item  related  training  data  exist  or  will  new  documentation  be  needed? 

•S  Will  the  NDI/commercial  item  need  repackaging  to  fit  the  environmental  profile? _ 
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3.4  RFP  Generation 

Once  the  requirements,  performance  specification,  and  acquisition  strategy  have  been  defined, 
documented,  and  approved,  the  next  step  is  transferring  this  data  into  a  RFP  consisting  primarily  of  a  SOW, 
performance  specification,  and  associated  CDRLs.  The  type  of  information  addressed  in  the  RFP  depends  upon  the 
type  of  acquisition  strategy  selected.  A  common  approach  may  be  for  a  prime  integrator  to  out  source  and  integrate 
commercial  items  into  an  end-item.  Potential  sources  of  assistance  in  generating  RFP  related  information  include: 

•  Acquisition  policies,  directives,  and  documentation  requirements  are  addressed  in  DoDD  5000.1,  DoD 
5000.2-R,  and  the  draft  SECNAVINST  5000.2B. 

•  Additional  assistance  can  be  found  in  the  Naval  Sea  Systems  Command’s  (NAVSEA)  electronic  Master 
Acquisition  Program  Plan  (MAPP)  and  on-line  via  the  Internet  and  the  acquisition  deskbook. 

•  The  SYSCOMs  also  have  acquisition  reform  agents  to  assist  program  offices  and  acquisition  engineers  in 
generating  and  staffing  the  RFP. 

The  generation  and  solicitation  processes  are  not  substantially  different  for  acquiring  open  system 
products.  DoD  is  simplifying  the  acquisition  process  and  compressing  schedules  to  achieve  cost  savings. 
Additionally  DoD  is  fostering  more  commercial  participation.  The  intent  is  to  reduce  restrictions,  eliminate  large 
volumes  of  redundant  documentation  and  delivery  items  that  describe  the  end  item.  In  cases  of  open  system 
commercial  items,  availability  of  DoD  unique  documentation  should  not  normally  be  required  in  today’s 
acquisition  environment.  The  contractor  is  now  given  more  latitude  in  how  documentation  and  product  descriptive 
data  is  presented  and  forwarded  to  the  government.  The  key  RFP  documentation  is  the  performance -based 
specification,  SOW,  and  CDRL  items,  but  these  documents  should  be  developed  expressing  the  requirements  and 
documentation  for  an  open  systems  acquisition.  Depending  on  the  acquisition  strategy  selected  (full  development, 
some  mixture,  or  all  NDI/commercial  item  products),  the  degree  and  content  of  technical  and  programmatic 
documentation  required  will  vary.  In  each  case  the  main  factor  that  must  be  considered  is  how  well  is  the  open 
systems  interface  requirements  and  related  data  needs  are  defined  in  the  solicitation  package.  A  draft  release  of  the 
RFP  should  help  the  program  managers  find  contractors  who  either  have  an  open  systems  experience  or  at  least 
understand  the  OSE  prior  to  a  final  RFP  release.  The  RFPs  should  provide  incentives  for  contractors  to  provide 
and  maintain  an  open  systems  approach.  A  program  acquisition  striving  to  achieve  an  OSE  should  take  a 
cumulative,  life-cycle  approach  to  open  systems  and  not  a  quick  fix  approach. 

The  IPT  should  play  a  major  role  in  the  development  of  the  RFP.  not  only  in  the  area  of  specification 
generation  but  also  in  development  of  the  SOW  and  CDRFs.  The  SOW  should  address  such  items  as:  requirement 
of  the  contractor  to  provide  an  open  system  implementation  and  migration  plan,  identification  of  the  contractors 
process  for  conducting  a  market  survey  and  how  the  survey  results  will  be  presented,  escrow  account(s) 
considerations  for  selected  products,  and  IPPD/IPT  involvement.  When  developing  the  RFPs  section  C, 
Instructions  to  Offerers,  a  requirement  should  be  inserted  for  the  bidder  to  provide  evidence  of  open  system 
experience  and  understanding  or  provide  a  sample  task  that  requires  the  bidder  to  respond  to  sample  task  using  an 
open  system  solution.  The  instructions  should  also  have  the  bidder  identify: 

•  Contractor's  opinions  and  process  for  standards  profile  applications  and  conformance  in  their  design. 

•  How  the  design  is  an  open  systems  architecture. 

•  Contractor's  life  cycle  support  strategy. 

•  Technology  refreshment  program  being  employed. 

•  Adherence  to  an  open  systems  approach. 

•  Strength  of  market  knowledge. 

•  Contractor's  conformance  management  approach. 


37 


NGCR  Document  No.  AST  003  ver  1 


31  December  1996 


Figure  3-10  provides  a  synopsis  of  the  process  for  integrating  the  performance  requirements  and 
acquisition  strategy  into  the  RFP. 


Error!  Switch  argument  not  specified. 


Figure  3-10  Process  Flow:  RFP  Generation 


If  the  procurement  comprises  a  full  application  of  NDI/commercial  item  products,  the  RFP  should 
specifically  address  the  integration  of  these  products,  conformance  and  interoperability  testing  verification  to 
selected  standards,  and  supportability  issues.  No  matter  what  the  degree  of  NDI/commercial  item  integration, 
compliance  to  internal  and  external  open  system  interface  standards,  along  with  conformance  and  interoperability 
testing  is  essential.  The  main  parts  of  a  RFP  (SOW,  CDRL.  and  performance  specification)  drive  the  acquisition 
effort  in  terms  of  cost  and  time.  Historically,  a  detailed  SOW  and  a  build-to  type  specification  provided  definitive 
information  for  system  development  based  on  Federal,  DoD  and  Military  Standards  requirements.  In  today's  open 
system  acquisitions,  the  emphasis  is  on  using  commercial  standards  and  specifications.  The  process  for  developing 
the  main  components  of  an  RFP  for  an  open  system  acquisition  is  discussed  in  the  following  sub-sections. 

3.4.1  Performance  Specification 

A  performance  specification  defines  the  functional  requirements  for: 

•  the  item  and  the  environment  in  which  it  must  operate, 

•  the  interface  and  how  it  will  be  tested,  and 

•  the  interchangeability  characteristics. 
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A  good  performance  specification1,  contains  requirements  that  are  quantitative  rather  than  qualitative, 
verifiable,  and  material  and  process  independent.  DoD  has  defined,  as  part  of  the  acquisition  streamlining  effort, 
the  order  of  precedence  of  specifications  for  DoD  acquisitions  (figure  3-11). 


GROUP  I 

Documents  mandated  by  law  or 
regulation  pursuant  to  law 

GROUP  II 

Performance  Documents 

GROUP  in 

Detailed  documents 

GROUP  IV 

Standards,  specifications,  and  related 
publications  issued  by  the  government 
outside  the  military  or  federal  series  for  the 
non-repetitive  acquisition  of  developmental 
items 

•  OSHAorEPA 
regulations 

•  Non-go vemment  standards 

•  Commercial  Item  Description 
(CID)  * 

•  Federal  specifications 

•  Standard  performance 

•  Non-government  standards  ** 

•  Federal  specifications 

•  Detailed  specifications  *** 

•  Purchase  descriptions 

•  Product  descriptions 

•  Program  peculiar  or  system 
specifications 

*  By  definition  a  CID  is  a  performance  specification. 

**  Non-government  standards  are  not  necessarily  performance-based.  They  should  be  examined  individually  to  determine  if  they  are 
performance-based. 

***  The  application  of  detailed  specifications  and  military  standards  requires  a  waiver. 

Figure  3-11  Specifications  Order  of  Precedence 


Depicted  in  figure  3-12  is  a  high  level  overview  of  a  typical  open  system  based  specification  development 
process.  The  operational  need  forms  the  basis  of  the  start  of  a  open  systems  based  performance  specification.  A  set 
of  derived  requirements  are  identified  from  the  operational  need  (MNS  and/or  ORD)  and  a  set  of  derived 
requirements  is  generated.  At  this  point,  a  minimum  list  of  interface  standards  can  be  developed  at  both  the 
Service  and  Joint  level,  thus  determining  the  initial  functional  requirements  that  require  specification  in  the 
contract.  As  part  of  the  performance  specification  development  process,  program  offices  need  to  evaluate  interface 
standards  in  terms  of  their  functional  profile,  determine  the  minimum  set  of  mandatory  requirements,  and  assess 
the  optional  profiles  and  associated  extensions  to  determine  usability  and  interoperability.  The  performance 
specification  should  clearly  identify  the  standards  selected  and  the  profiles  necessary  to  achieve  conformance. 

Selected  standard  profile(s)  should  be  addressed  in  the  performance  specification  to  the  degree  necessary 
to  ensure  enough  information  is  passed  to  the  contractor  to  adequately  perform  conformance  testing,  achieve 
interoperability,  compatibility,  scaleability,  and  performance.  The  standards  and  their  profiles  integrated  into  the 
specification  should  be  compared  to  the  architecture  requirements  of  the  acquisition  program  (both  at  the  Service 
and  Joint  levels)  and  modified  as  required  to  insure  interoperability  and  compatibility.  At  this  point  the  program 
office  can  decide  whether  other  Service  NDI/commercial  item  products  can  be  of  use  and  identify  any  discrepancies 
between  standards  selection  and  profile  restrictions.  Also  an  initial  market  survey  should  be  performed  to 
determine  implementation  and  migration  opportunities  of  commercial  items  to  meet  the  functional  needs  of  the 
design.  This  process  should  greatly  reduce  risks  in  terms  of  performance,  interoperability,  and  in  the  long  run 
cost-effectiveness  and  schedule  risks.  The  results  of  the  market  survey  should  be  used  to  refine  the  RFP 
documentation  especially  the  performance  specification.  Sufficient  information  should  be  available  to  start 
determining  if  sufficient  products  are  available  in  the  market  place  and  if  they  have  the  support  to  meet  the 
programs  requirements  or  whether  a  full  or  partial  development  effort  is  needed.  The  specification  may  need  to  be 
refined  to  address  those  portions  of  the  design  that  need  development.  The  test  requirements  section  of  the 
specification  should  likewise  address  NDI/commercial  item  applications  to  the  design.  No  matter  how  the 
functions  are  achieved  in  the  design,  the  need  to  adequately  test  for  interface  conformance  remains  a  critical  part  of 
the  specification  requirements. 


SD-15,  Performance  Specification,  OASD,  29  June  1995. 
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Figure  3-12  Process  Flow:  Performance  Specification  Development 


Typical  considerations  a  program  office  evaluates  when  generating  a  program's  performance  specification 
are  provided  below. 


PERFORMANCE  SPECIFICATION  CONSIDERATIONS 

Have  requirements  been  specified  so  as  to  promote  full  and  open  competition? 

Have  the  minimum  requirements  or  thresholds  been  stipulated? 

Have  thresholds  been  defined? 

Have  constraints  been  identified  and  addressed  within  the  specification  in  a  clear  manner? 

•S  Are  requirements  stated  in  a  quantitative  versus  qualitative  manner? 

Have  specifications  been  stated  in  terms  of  function  and  performance  vs.  ‘  How  To”,  and  design  requirements 
(environment,  supportability,  etc.)? 

Are  interfaces  defined  in  sufficient  detail  to  allow  interchangeability? 

•  Standards  and  standard  profiles  (mandatory  and  optional  features  used  by  the  system)? 

•  Interface  conformance  (testing,  test  reports)  incorporated  into  T&E  efforts? 

Was  the  specification  developed  using  market  research  in  order  to  leverage  commercial  products? 

Have  the  standards  and  standard  profile  needed  to  meet  the  derived  set  of  requirements  been  checked  against 
systems/products  that  must  interoperate  to  ensure  no  unique  application  of  the  interface  exists  elsewhere  in  an 
architecture  that  could  compromise  interoperability? 

S  Does  the  specification  meet  acquisition  streamlining  criteria? _ 


3.4.2  Statement  of  Work  (SOW) 

The  SOW  or  statement  of  objectives  (SOO)14  (hereafter  referred  to  as  SOW),  along  with  a  performance 
specification  and  associated  CDRLs,  provides  a  definitive  scope  of  effort  for  the  system  developer/manufacturer. 
The  SOW  defines  the  types  of  information/documentation  required;  types  of  reviews  to  be  conducted;  the  testing 
required;  test  data  requirements  (traceable  to  the  performance  specification  and  test  and  evaluation  master  plan 
(TEMP));  and  logistics  requirements  (logistics  support  analysis  (LSA),  reliability/maintainability/availability 
(RMA),  training,  supply  support  requirements,  etc.).  OSE  requirements  are  addressed  to  include  the  use  of 
commercial  standards  when  feasible.  The  SOW  also  contains  contractor  requirements  for  configuration 
management,  and  quality  assurance,  along  with  program  management  requirements,  policies,  and  procedures. 


It’s  imperative  that  in  an  open  systems  acquisition  that  the  SOW  clearly  identify  tasks  such  as: 


14  MIL-HDBK-245D,  3  APR  96. 
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•  establishment  of  a  conformance  management  program, 

•  identifying  and  generating  data  requirements  for  product  integration, 

•  product  CM  planning  and  tracking,  and 

•  defining  how  CM  and  quality  assurance  (QA)  requirements  will  be  invoked  on  vendor/  subcontractors. 

•  identify,  establish,  and  implement  design/baseline  control  methods/processes 

•  define  OSE  related  requirements  including  logistics  and  supply  support  concepts  and  methods  for 

transferring  and  accessing  logistics/support  data. 

The  integrator,  whether  a  DoD  or  contractor  agency,  will  need  to  combine  data  from  several  different 
products  into  usable  training  and  maintenance  documentation.  Operator  and  maintenance  manual  requirements 
can  be  leveraged  from  data  already  available  from  commercial  producers.  The  data  integration  effort  can  be  part  of 
a  solicitation  and  priced  for  either  the  Integrator/Prime  contractor  or  government  agency  as  warranted. 

Requirements  for  the  development  and  support  of  training  data  and  devices  needs  to  be  stated  in  the  SOW. 
In  some  cases,  the  major  issue  to  consider  for  including  a  work  task  in  the  SOW  may  be  to  generate  the  training 
data  for  an  aggregate  set  of  products  from  a  multitude  of  OEMs  that  comprise  the  design. 

The  SOW  has  to  include  requirements  for  NDI/commercial  item  analysis  and  trade-off  studies.  This 
includes  performance,  product  availability,  conformance,  and  interoperability  test  data  (including  simulation  and 
modeling  results),  design  criteria  and  associated  interface  standards,  and  production  line  plans  for  each  vendor  and 
product.  Other  SOW  considerations  are  requirements  for  the  contractor  to  provide  the  standards/profiles  for  each 
selected  interface;  thereby  forming  a  minimum  set  of  profiles  that  baseline  the  design.  Typical  open  system  related 
SOW  considerations  are: 


STATEMENT  OF  WORK  CONSIDERATIONS 

•S  Does  the  SOW  encompass  the  open  systems  program  requirements  for  qualifying  commercial  items? 

S  Do  requirements  exist  for  the  contractor  to  establish  a  conformance  management  program? 

Does  the  conformance  management  program  addresses  how  the  contractor  will: 

•  fix  non-conforming  behavior  or  restrict  other  components  use  to  features  that  do  conform; 

•  require  engineer  conformance  into  anything  that  will  be  developed;  and. 

•  document  and  manage  by  the  specific  interfaces  selected? 

Have  open  system  product  Level  of  Repair  (LOR)  requirements  been  addressed? 

Have  sparing  requirements  for  NDI/commercial  items  been  addressed? 

Have  all  work  efforts  been  stipulated  for  defining,  performing,  and  documenting  all  necessary  OSE  related  test 
requirements  specified  in  the  performance  specification  including: 

•  interoperability  testing;  and. 

•  conformance  testing? 

^  Is  there  a  requirement  for  the  prime/integrator  to  invoke  commercial  CM  and  QA  standards  where  applicable? 
^  Is  documentation  of  standards  profiles  applications  required  and  are  specific  contractual  mandates  addressed? 
•S  Is  test  data  verification  required  (vs.  claimed  compliance)? 

•  Are  third  party  test  results  available? 

•  Does  the  test  data  directly  correlate  to  product  in  question  or  to  product  line? 

•  Do  the  test  data  results  provide  interface  profile  conformance  or  compatibility  verification? 

As  components  evolve,  are  techniques  for  assessing  their  interface  conformance  addressed? 

If  depot  level  support  is  required  are  tasking  and  periods  of  performance  clearly  defined? 

S  Have  technical  manual  and  training  requirements  been  identified  for  commercial  item  integration  support? 

•S  Does  the  SOW  address  contractor  scheduling  and  planning? 

•S  Does  the  SOW  address  reliability  and  risk  management  planning  and  metric  task  efforts? _ 

3.4.3  CDRL  (Contract  Data  Requirements  List) 
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Data  item  deliverable  cost  factors  can,  in  some  cases,  overshadow  the  actual  product  costs.  DoD 
streamlining  and  standardization  efforts  have  focused  on  the  cost  versus  data  value  issue  and  evaluated  the  depth 
and  complexity  of  the  current  CDRL  package  versus  cost  effectiveness.  The  focus  has  shifted  to  acquiring  the  right 
amount  of  data  to  ensure  a  cost-effective  and  performance  capable  product  that  is  supportable  and  based  on 
commercial  standards,  specifications,  and  CIDs.  The  program  office  needs  to  tailor  deliverables  to  fit  need,  and 
where  possible,  to  fit  widely  accepted  commercial  specifications,  standards,  and  processes.  Only  essential  data 
requirements  to  baseline  and  control  products  should  be  required  of  the  contractor.  The  program  office  should 
review  data  requirements  and  the  contractor's  format  wherever  feasible.  The  following  checklist  consists  of 
typical  questions  that  need  to  be  asked  when  generating  a  CDRL  package  for  an  OSE  acquisition.  The  CDRL 
package  should  be  generated  in  conjunction  with  the  development  of  the  open  system  based  performance 
specification  and  associated  SOW. 


CDRL  CONSIDERATIONS 

S  Have  DIDs  associated  with  commercial  standards  and/or  specifications  been  used? 

Have  DIDs  been  tailored  to  allow  industry  involvement? 

Does  the  CDRL  package  address  all  OSE  related  SOW  work  areas  such  as: 

•  configuration  management  and  control  of  baseline/reporting  requirements, 

•  conformance  management  plans, 

•  technical  data  including, 

•  vendor  documentation, 

•  documentation  of  standard  profile(s), 

•  interface  and  interface  conformance  data, 

•  commercial  item  operator  and  maintenance  manualfs), 

•  interface  features  used  by  the  system,  i.e.,  standard  (mandatory  and  optional)  features  used  by  the 
system, 

•  logistic  support  analysis  reports, 

•  quality  assurance  plan  and  processes, 

•  test  and  evaluation  plans  and  reports  (conformance,  interoperability,  portability,  scaleability, 
acceptance,  and  regression  testing,  etc.), 

•  training  plans  and  reports, 

•  maintainability  and  maintenance  data, 

•  reliability  data  and  documentation  (analysis  test  results,  e.g.,  finite-element  and  boundary-element 
analysis  (FEA/BEA)  assessments),  and 

•  electronic  data/documentation  requirements/reports? 

Have  sparing  requirements  and  data  been  defined  and  documentation  incorporated  in  the  CDRL  package 
(important  in  mixed  and  full  development  efforts)? 

•S  Have  adequate  requirements  been  defined  for  repairable  items? _ 


3.5  Contract  through  Deployment 

The  Navy  is  rapidly  changing  to  an  evolutionary  acquisition  approach  which  includes  cyclical  acquisitions 
incorporating  upgrade/field  changes  or  new  systems  by  deployment  battle  groups.  To  field  a  system  that  meets 
operational  needs  in  a  cost  effective  and  supportable  manner,  program  offices  and  associated  IPTs  will  need  to 
constantly  track  and  update  the  program  profiles  and  requirements  to  fit  emerging  technology  and  product  line 
variations.  Once  a  program  has  adequately  defined  its  requirements  and  generated  the  technical  documentation  to 
procure,  the  RFP  is  issued  and  a  process  put  in  place  to  evaluate  responses  to  the  RFP.  The  down-selection  of 
submitted  proposals  in  terms  of  an  open  systems  acquisition  versus  a  traditional  full  development  acquisition 
requires  attention  to  different  discriminators  than  in  past  acquisitions.  The  Technical  Evaluation  Board  (TEB), 
normally  convened  to  evaluate  the  proposals,  should  now  include  the  IPT  to  ensure  the  OSE  and  related  standards 
knowledge  base  is  available  to  properly  evaluate  the  proposals. 
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The  process  for  moving  the  program  from  the  high  level  performance  specification  to  a  fielded  system  is 
depicted  in  figure  3-13. 


Figure  3-13  Process  Flow:  Contract  Award 


The  evaluation  by  the  TEB  needs  to  focus  not  just  on  the  prime  contractor/integrator  but  on  the 
commercial  items  that  make  up  the  system/equipment.  Systems  will  consist  of  multiple  OEM  products  integrated 
into  an  overall  design,  so  the  depth  of  evaluating  a  proposal  may  necessitate  suppliers  and/or  vendors  participation. 
Access  to  and  value  of  available  technical  data  will  be  a  premium  factor.  Compliance  to  the  standards  of  the 
external  interfaces  of  the  final  assembly  and  the  internal  interfaces  of  the  commercial  items/LRUs  that  make  up  the 
product  is  very  important.  Schedule,  commercial  item  availability,  and  support  items  are  essential  in  the  decision 
process.  The  ability  to  track  all  the  commercial  items  that  make  up  the  final  end  item  in  terms  of  interface 
application,  changes,  product  line  modifications,  and  interoperability  are  likewise  essential. 

Some  items  of  consideration  when  selecting  a  contractor  or  integrator  for  an  OSE  program  effort  are 
provided  below. 


CONTRACTOR  SELECTION  CONSIDERATIONS 

•S  What  is  the  contractor  and  vendor's  past  performance  in  the  OSE? 

Is  a  quality  assurance  process  in  place  for  both  contractor  and  vendors? 

Has  the  contractor  specified  an  adequate  method  to  measure  conformance  to  standards? 

•  Are  appropriate  alternatives  in  place  to  handle  correction  of  non-conformance? 

•S  Does  the  contractor/integrator  have  a  method  in  place  to  track  products  and  designs  as  it  relates  to  conformance  to 
the  standards  and  standard  profiles? 

Are  vendors  capable  of  OEM  repair?  What  is  their  history  in  meeting  repair  or  warranty  requirements? 

•S  Does  the  contractor  have  a  plan  to  track  interface  applications  integrated  into  the  system  over  a  period  of 
performance? 

•S  Has  conformance  testing  been  conducted  on  parts/components/LRUs  that  are  in  use  commercially  and  which 
fulfill  functional  requirement  and  interface  standards  ? 

•S  Does  the  current  production  output  meet  the  program  needs? 

•  How  much  of  the  market  share  is  the  system  need? 

•  Is  the  program  the  only  reason  for  a  new  production  run? 

•  Are  the  products  real  (not  vaporware)? 

•S  Have  the  product  update  cycles  and  support  time  been  defined? 

S  If  NDI/commercial  items  are  being  integrated,  have  the  life  cycle  support  times  been  established? 

•S  Does  the  choice  of  open  system  interface  standards  products  influence  out-year  support  ? 

•S  Are  there  second  sources  of  products  for  the  functions  being  profiled  ? 

Have  built-in  test  and  built-in  test  equipment  (BIT/BITE)  requirements  been  addressed? _ 


43 


NGCR  Document  No.  AST  003  ver  1 


31  December  1996 


CONTRACTOR  SELECTION  CONSIDERATIONS 

•S  Are  computer  aided  design  and  computer  aided  manufacturing  (CAD/CAM)  drawings  available?  Does  electronic 
access  to  vendors  CAD/CAM  exist? 

Is  reliability  data  available? 

•S  Are  technical  manuals  available  and  do  they  meet  program  requirements? 

•S  Are  warranties  to  be  provided?  Do  they  meet  program  needs? 

S  Have  training  considerations  been  met? 

•S  Have  all  environmental  profile  requirements  been  addressed  and  can  they  be  met? 

•S  Have  all  supportability  profile  requirements  been  addressed  and  can  they  be  met? 

V  Does  the  contractor  have  a  plan  to  mitigate  the  risk  factors  identified  as  part  of  the  proposal  review? _ 


3.6  Contract  Execution 

Figure  3-13  identifies  a  contract  execution  function  perforated  prior  to  deploying  an  OSE  related  product. 
This  function  and  associated  processes  are  represented  in  Figure  3-14.  The  figure  illustrates  that,  upon  contract 
award,  well  defined  systems  engineering  efforts  should  be  implemented  by  the  contractor  (with  the  IPT  acting  as 
oversight)  in  order  to  ensure  proper  transition  of  the  OSE  acquisition  from  a  design  and  integration  state  effort  to  a 
deployable  system.  As  part  of  the  source  selection  process  and  prior  to  executing  the  OSE  oriented  contract 
package,  the  government,  using  available  IPT  expertise,  should  have  evaluated  the  contractor's  capabilities  and  the 
past  performance  of  proposed  and  associated  LRU  vendors.  Considerations  should  have  also  included  the  proposed 
contractor/manufacturer’s  reliability  and  supportability  factors,  availability  of  test  conformance  data  for  proposed 
NDI/commercial  items,  and  identification  of  any  migration  options  of  the  proposed  design.  Upon  applying  the 
various  weighting  factors  of  the  selection  criteria,  a  contract  is  awarded  and  execution  plans  put  into  place. 

No  matter  which  acquisition  strategy  was  selected  or  who  is  the  contract  performer,  a  solidification  of  the 
design  and  associated  interface  standards,  standards  profiles,  and  selected  solution  of  products  needs  to  be 
achieved.  In  some  cases,  the  selected  contractor  (or  the  government  in  some  acquisition  scenarios)  may  find 
factors  have  changed,  such  as: 

•  Commercial  items  originally  selected  may  no  longer  be  supported. 

•  Technology  advancements  may  have  been  over  taken  events. 

•  Product  manufacturing  or  sparing  capabilities  may  no  longer  exist  or  meet  program  life  cycle 
schedules. 

•  Product  lines  may  no  longer  be  in  compliance  or  compatible  do  to  changes  of  internal  design. 

•  Funding  profiles  may  have  necessitated  a  change  of  performance  requirements. 

•  Vendor  or  supplier  availability  could  have  changed. 

The  contractor,  working  in  conjunction  with  the  government  (IPT),  should  use  the  market  survey  and 
trade-off  analyses  (including  cost)  process  to  verify  that  the  agreed  upon  approach  and  design  is  the  best  value  in 
terms  of  functional  performance,  supportability,  and  cost-effectiveness. 

The  contractor  needs  to  verify  to  the  government  that  the  selected  NDI/commercial  items  being  integrated: 

•  meet  all  specified  requirements; 

•  are  products  that  conform  and  are  compliant  to  the  specified  interface  standards  and  standards  profiles; 

and 

•  have  conformance  test  data  provided  to  the  government  or  conduct  tests  to  ensure  conformance  and 

compatibility  prior  to  acceptance  of  the  system/product  by  the  government. 

If  the  test  data  does  not  exist  or  contractor  test  capabilities  are  insufficient  to  verify/validate  conformance 
and  compatibility  of  the  product(s),  then  either  the  contractor  or  the  government  must  identify  the  facilities  that 
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can  perform  the  verification/validation  of  the  product(s).  It  is  important  that  the  interface  standard(s),  test 
procedures,  and  associate  test  results  be  well  documented  to  establish  the  baseline  for  the  deployed  system. 

Any  shortfalls  identified  in  finalizing  the  design  must  be  documented  and  addressed.  Solutions  rectifying 
the  shortfalls  must  be  implemented  prior  to  proceeding  to  acceptance  and  deployment.  Potential  solutions  include 
replacing  products  that  no  longer  are  available  with  a  product  that  meets  the  specified  form,  fit,  and  function 
factors  but  may  not  have  been  selected  because  of  being  more  costly,  having  fewer  optional  features  than  the 
previous  product,  or  having  less  technical  or  supportability  data  (i.e.,  technical  manual)  available.  This  may 
require  the  program  office  and  the  IPT  revisit  the  specified  requirements  (including  the  ORD)  to  see  if  a  relaxation 
of  the  specified  requirements  or  programmatic  needs  could  be  permitted  to  allow  use  of  the  product  without 
compromising  the  system’s  objectives  and  thresholds. 


Once  the  baseline  is  finalized  and  integration  and  testing  is  complete,  the  product  can  transition  into  life 
cycle  maintenance.  The  contract  execution  function  should  start  the  efforts  to  put  supportability  factors  and 
processes  in  place.  These  include  but  are  not  limited  to  depot  support,  sparring,  implementation  of  the 
configuration  management  plan,  and  tracking  of  the  baseline  and  technology  factors. 
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Figure  3-14  Process  Flow:  Contract  Execution 
3.7  Life  Cycle  Maintenance  and  Upgrades 


45 


NGCR  Document  No.  AST  003  ver  1 


31  December  1996 


NDI/commercial  item  product  obsolescence  often  drives  how  maintenance  is  accomplished.  Maintenance 
is  more  likely  to  come  in  the  form  of  product  upgrade.  Market  surveillance  of  OSE  products  and  emergent  OSE 
technologies  is  necessary  for  support  and  upgrade  of  systems  in  all  environments.  In  many  cases,  it  may  be  helpful 
to  closely  monitor  the  commercial  standards  being  developed.  Figure  3-15  provides  an  overview  of  the  life  cycle 
maintenance  and  upgrade  process. 


OEM  DEPOT 
WARRANTIES 
SPARES 


Figure  3-15  Process  Flow:  Life  Cycle  Maintenance  and  Upgrades 


The  initiation  of  the  life  cycle  requirements  addressed  in  figure  3-15  starts  at  the  very  beginning  of  the 
open  systems  acquisition  process  as  part  of  the  development  of  the  MNS/ORD  functional,  environmental, 
supportability,  and  product  requirements.  As  the  functional  profile  is  derived  from  the  MNS/ORD,  so  are  the 
environmental  and  supportability  profiles.  As  the  design  and  standards  selection  are  refined  and  products  are 
evaluated,  so  are  the  life-cycle  support  issues.  A  major  part  of  the  market  survey  is  in  the  area  of  product 
supportability  of  the  technologies  and  products  under  consideration.  The  key  to  successful  reduction  of  program 
risks  is  how  well  life  cycle  factors  are  predicted  and  planned. 

Traditionally  DoD's  approach  was  to  define  the  unique  interfaces  and  processes  in  a  very  detailed  critical 
or  prime  item  developmental  or  product  specification  that  required  extensive  testing  to  verify  function  performance 
and  quality  control  for  the  production  and  support  of  the  end  item.  The  components  at  both  the  piece  part  and  end 
item  level  were  usually  developed  from  scratch  or  unique  enough  to  require  extensive  out  year  supply  support  and 
individual  program  related  training  for  operators  and  maintenance.  Parts  support  for  a  full  development 
acquisition  were/are  usually  not  cost-effective  or  usable  in  other  systems/program  efforts.  The  open  system  life 
cycle  support  stems  from  the  use  of  standards  that  are  being  used  in  the  market  place  for  a  variety  of  uses.  The 
hardware  and  software  components  that  implement  the  standard(s)  can  be  acquired  and  integrated  to  meet 


46 


31  December  1996 


NGCR  Document  No.  AST  003  ver  1.0 


functional  needs  of  the  system.  The  supply  support  for  the  OSE  system  component  products  will  generally  be 
available  for  a  shorter  period.  The  length  of  available  supply  support  will  be  driven  by  the  commercial  market 
place  and  rate  of  product  change.  Stockpiling  components  is  not  usually  a  cost-effective  approach.  As 
commercial  items  are  more  freely  integrated  into  the  main  stream  of  DoD  systems,  the  supply  support, 
documentation,  and  training  support  requirements  will  have  to  keep  pace  with  the  commercial  marketplace.  OSE 
related  training  requirements  (versus  established  Navy  Training  techniques)  may  need  to  be  closely  evaluated  and 
tailored.  Training  such  as:  computer  based  training  (CBT),  video  teleconferencing  classes  (i.e.,  upgrades),  video 
cassettes,  and  other  types  of  media  should  be  considered  in  addition  to  or  in  lieu  of  traditional  training  classes. 
Training  by  standards  and  associated  fielded  products  (i.e.,  software  packages)  may  also  be  appropriate.  Section  4 
discusses  in  more  detail  the  open  system  considerations  that  should  be  addressed  when  acquiring  an  open  systems 
product. 


Some  considerations  when  planning  life-cycle  maintenance  and  upgrades  support  for  an  open  systems  are 
provided  below. 


LIFE  CYCLE  MAINTENANCE  and  UPGRADE  CONSIDERATIONS 

•S  Does  the  current  life-cycle  maintenance  system  sufficient  meet  the  new  requirements? 

Is  the  maintenance  concept  sufficient  to  meet  OSE  needs? 

•S  Are  the  training  requirements  already  in  place  capable  of  meeting  open  systems  requirements? 

•  Are  the  commercial  item  manuals  sufficient  for  operator  and  maintainer  requirements? 

•  Is  contractor  training  available?  Is  it  cost-effective? 

•  Is  embedded  training  available  and  is  it  part  of  the  design? 

•  Is  CBT  an  option  and  available? 

•S  Is  technical  data  available  in  digital  form?  Is  it  required  as  part  of  the  contract? 

Is  the  warranty  adequate? 

•  If  modifications  are  required,  is  the  warranty  affected? 

•  Does  a  warranty  need  negotiation  ? 

•S  Is  the  depot  providing  adequate  and  timely  repair?  Is  it  cost-effective? 

^  Is  commercial  support  more  cost-effective? 

•S  Are  supply  support  plans  meet  OSE  requirement? 

^  Is  an  escrow  account  available? 

Have  life-cycle  maintenance  impacts  been  addressed  for: 

•  interface  standards  stability, 

•  product  application  and  stability  of  interfaces, 

•  technology  and  standards  development/revisions, 

•  Joint  Service  application,  and 

•  changes  to  system  interface  standard  profiles  (external  and  internal)? 

•S  Have  leased  options  for  support  been  considered  and  is  it  a  cost  effective  option? 

•S  Are  other  Service/Joint  programs  using  the  same  commercial  items? 

•S  Are  other  Service/Joint  program  maintenance  support  capabilities  for  these  commercial  items  available? 

Is  conformance  tracking  plans  in  place  and  being  implemented? 

S  Do  related  OSE  technical  changes  (i.e.,  engineering  change  proposals  (ECPs),  specification  change  notices 
(SCNs),  etc.)  consider  life -cycle  support  issues? 


4.0  SUPPORTABILITY 
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Department  of  Defense  (DoD)  Regulation  5000.2-R  requires  that  supportability  factors  be  stated  as 
performance  requirements  in  the  contract.  This  section  discusses  the  underlying  integrated  logistics  support  (ILS) 
elements  and  related  disciplines  necessary  to  develop  these  performance  specifications  for  support  in  the  contract. 
The  discussion  focuses  on  the  changes  in  approach  and  implementation  required  in  an  open  system  environment 
(OSE).  This  section  provides  some  insight  necessary  to  implement  a  minimum  risk  open  system  support  system 
consistent  with  Navy  acquisition  policy.  The  disciplines  discussed  are  provided  in  table  4-1. 


DISCIPLINE 

REFERENCE  to  DoD  5000.2-R 

Design  Interface 

Paragraph  4.3.4 

Maintenance  Planning 

Paragraph  4.3.3. 1  and  4. 3. 3. 2 

Manpower  and  Personnel 

Paragraph  3.5.2 

Supply  Support 

Paragraph  4. 3. 3. 4 

Support  Equipment 

Paragraph  4. 3. 3. 4 

Technical  Data 

Paragraph  3. 3. 4. 5  and  4. 3. 3. 3 

Training  and  Training  Support 

Paragraph  4. 3. 3. 4 

Table  4-1  ILS  Disciplines  with  Reference 


The  disciplines  are  discussed  in  this  section  in  terms  of  considerations  that  need  to  be  addressed  when 
conducting  an  open  system  interface  standards  based  acquisition.  Discussions  regarding  open  system 
supportability  approaches,  methodologies,  and  recommendations  address  the  unique  aspects  of  an  open  system 
interface  standard  acquisition. 

When  using  open  system  interface  standards,  a  best  value  approach  should  be  pursued  to  balance  cost, 
performance,  schedule,  operational  readiness,  and  supportability.  The  use  of  open  system  interface  standards 
promotes  an  environment  in  which  interface  conformant  products  from  multiple  original  equipment  manufacturers 
(OEMs)  can  be  integrated  to  form  functional  systems.  Supportability  issues  must  be  part  of  the  criteria  evaluated 
during  the  selection  of  the  system  architecture.  It  is  imperative  that  detailed  product  support  planning  occur 
concurrently  with  the  start  of  the  development  process.  Commercial  product  baselines  are  continually  migrating 
because  industry  is  releasing  new  products  every  18-24  months.  Support  must  be  defined,  planned,  and  purchased 
for  each  commercial  product  baseline  change  incorporated  into  a  DoD  system. 

4.1  Design  Interface 

In  a  traditional  development  program,  design  interface  started  as  the  design  was  developed  and  tended  to 
consist  largely  of  logistic  support  analysis  (LSA)  tasks  and  participation  in  formal  MIL-STD-1521  design  review 
activities.  Rarely  did  the  ILS  Manager  or  ILS  staff  have  opportunity  to  analyze  potential  designs  and  compare 
their  inherent  supportability  parameters,  constraints,  and  costs  in  a  timely  enough  manner  to  affect  them 
significantly.  The  traditional  situation  was  waiting  for  the  design  engineers  to  be  reasonably  happy  with  the  result 
of  their  efforts  prior  to  release  of  drawings  to  the  “ilities”  team  for  their  initial  review.  Generally,  the  “ 
review  was  too  late  in  the  design  process  to  cost  effectively  impact  supportability  shortfalls.  Involvement  of  the 
ILS  team  usually  started  after  the  milestone  II  decision.  By  this  time,  about  85  percent  of  product  support  costs 
have  already  been  established  by  design  decisions.15 

In  an  open  system  environment,  choices  can  and  must  be  made  between  competing  architectural 
possibilities  prior  to  program  commitment  to  a  system  architecture,  a  set  of  standards,  and  standards  profiles. 
Once  the  architectural  decisions  have  been  made,  most  of  the  big  drivers  for  product  support  methods  and  costs  are 


15  James  V.  Jones,  LSA  Handbook  (Blue  Ridge  Summit,  PA:  TAB  Professional  and  Reference  Books,  1989),  17. 
Support  costs  determined:  70%  by  end  of  concept  phase;  85%  by  end  of  system  definition;  and  90%  by  the  end  of 
full  scale  development. 
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in  place.  The  architectural  decision  is  generally  a  technology  choice,  and  in  most  cases,  an  actual  product  type 
decision.  The  known  linkage  between  specific  standards,  profiles,  and  products,  tied  to  the  rapid  availability  of 
these  products  and  their  related  support  structures,  can  create  opportunities  for  substantive  timely  input  to 
architectural  decisions. 

Supportability  inputs  to  design  considerations  start  as  the  initial  requirements  are  being  established. 
Functional  reference  models  can  be  created  and  used  to  model  competing  architectures  and  supportability  options. 
Supportability  options  abound  until  the  final  architectural  decision  has  been  made.  Inputs  to  the  decision  process 
such  as  market  surveys,  architectural  comparisons,  and  tradeoff  activities  culminate  in  the  use  of  supportability 
analyses  and  corresponding  life  cycle  cost  (LCC)  forecasts.  The  principal  methods  of  influencing  system  design 
are  to  become  proactive  in  the  market  survey  and  trade-off  studies.16 

Once  the  system  architecture  has  been  selected,  the  design  interface  task  changes  to  ongoing  interaction 
with  the  engineers  participating  in  planning  and  implementation  of  system  refinements  and  support.  In  an  open 
system  environment,  evolutionary  acquisition  and  upgrade  as  an  element  of  product  support  are  expected  to 
become  normative.  Multiple  baselines  will  need  to  be  defined  and  supported.  The  design  will  change  as  will  the 
associated  products  and  standards.  Ongoing  interaction  among  the  ISEA/CFA,  SSA,  design  agent,  test  facility, 
and  others,  will  be  necessary  to  refine  and  enable  effective  support  and  upgrade  of  systems  at  all  user  locations. 
The  dynamic  nature  of  commercially  based  open  systems  requires  continuous  design  interface  throughout  the 
system  life  cycle. 

In  the  past  the  Navy  has  relied  on  build-to-print  and  source  control  drawings  to  procure  spares  and 
maintain  deployed  systems.  With  the  advent  of  frequently  changing  commercial  product  families  and  just-in-time 
procurements  the  roles  of  the  ISEA/CFA  and  SSA  become  more  pronounced.  The  use  of  specified  form,  fit,  and 
function  requirements,  performance  requirements  and  vendor  item  drawings  (VID)  will  become  more  prevalent. 
The  ISEA/CFA  and  SSA  (or  prime  contractor)  will  have  to  track  and  test  the  changing  product  family  members  as 
they  progress  from  one  revision  to  the  next  in  order  to  support  production  or  sparing  requirements.  The  VID 
should  be  monitored  to  ensure  that  it  includes  the  newest  product  in  the  evolving  product  family. 

Bridges  provide  pathways  for  compatibility.  They  give  the  ability  to  have  some  portions  of  a 
computer-based  product  use  a  legacy  architecture  (e.g.,  AN/UYK-44  or  Versa  Module  Europe  (VME)  products) 
while  upgrading  other  portions,  achieving  improved  functionality,  and  enabling  gradual  upgrade  of  the  computer 
resource  to  newer  and  more  powerful  OSA  standards.  The  bridged  products  within  an  equipment  or  system 
interoperate,  which  is  to  say  they  can  intelligently  exchange  data  and  not  interfere  with  each  other's  functionality. 
To  develop  bridges  between  DoD  unique  instruction  set  architectures  and  commercial  OSA  architectures,  DoD 
must  either  spend  development  dollars  or  offer  commercial  developers  the  opportunity  to  upgrade  a  large  enough 
number  of  DoD  unique  computers  to  justify  the  expenditure  of  private  sector  development  funds. 

If  DoD  is  to  leverage  its  dollars  (i.e.,  in  the  tactical  computer  resource  arena),  it  must  search  out  and  use 
cost-effective  opportunities  to  standardize  at  the  architectural  as  well  as  product  level,  not  just  within  DoD,  but 
also  with  the  commercial  marketplace.  If  the  commercial  marketplace  is  using  vast  quantities  of  products  that 
conform  to  OSA  standard  XYZ,  then  DoD  should  try  to  standardize  on  these  products  too.  The  result  will  be  ready 
availability  of  product  support  infrastructure  and  services,  and  timely  availability  of  upgrade  paths,  all  based  on 
commercially  based  investment. 


Typical  design  interface  considerations  are  listed  below. 


16  Refer  to  section  5.1  and  5.2  for  additional  information. 
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DESIGN  INTERFACE  PLANNING  CONSIDERATIONS 

■S  Involvement  of  IPT  in  all  facets  of  design  interface  considerations. 

■S  Maturity  of  the  architecture. 

■S  Identification  of  Standards,  standards  interfaces,  and  associated  selected  standards  profiles. 

■S  Determination  that  sufficient  data  exists  and  is  available  (e.g.,  detailed  profiles  and  test  data)  to  translate  the 
functional  needs  of  the  design  into  a  technical  reference  model  (TRM),  performance  specification,  and 
statement  of  work  (SOW). 

■S  Have  functional,  environmental,  supportability,  and  product  technical  requirements  been  addressed  by  the 
IPT? 

■S  Modeling  and  simulation  efforts  employed  that  evaluated  alternative  approaches/designs  and  verified 
interoperability  and  integration  factors  to  product  needs. 

■S  Cost  -effectiveness  studies  performed  prior  to  final  selection  of  design  interfaces. _ 
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4.2  Maintenance  Planning 


Maintenance  planning  is  the  process  that  evaluates  system  design  iterations  with  respect  to  the  program's 
mission,  operational  availability  (A0)  requirements,  and  costs  in  order  to  define  and  refine  the  maintenance 
concept.  In  a  developmental  program  that  uses  open  system  interface  standards,  maintenance  planning  is  the  same 
as  in  traditional  developmental  programs.  However,  for  open  system  interface  standards  based  non-developmental 
item  (NDI)  and  commercial  item  system  integrations,  the  maintenance  planning  task  is  to  ensure  cost  effective, 
efficient  product  support.  This  requires  innovative  and  creative  planning.  Figure  4-1  provides  a  theoretical 
overview  of  maintenance  planning  and  its  relationship  to  other  elements  of  supportability. 
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Figure  4-1  Maintenance  Planning  Overview 


The  use  of  open  system  interface  standards  and  associated  market  survey  (section  5.1)  makes  information 
available  for  use  earlier  in  the  maintenance  planning  process.  The  market  survey  information  includes: 

•  products  that  may  be  used, 

•  cost  of  product(s), 

•  how  the  products  are  supported, 

•  cost  of  support, 

•  availability  of  product(s)  in  market  place, 

•  contract  mechanisms  to  allow  ordering, 

•  availability  in  time  frame  needed,  and 

•  availability  of  product(s)  for  test  and  evaluation. 

This  information,  along  with  knowledge  of  the  open  system  interface  standards,  provides  the  supportability  team 
with  the  ability  to  conduct  effective  maintenance  planning. 

Because  information  is  readily  available,  the  initial  maintenance  planning  evaluation  task  can  include 
actual  what-if  and  at-what-cost  scenarios.  The  compressed  schedule  resulting  from  the  shorter  transition  from 
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concept  to  production  phase,  places  tremendous  pressure  on  the  maintenance  planning  function.  Thus  the 
maintenance  planner  must  also  become  a  proactive  member  of  the  acquisition  team.  A  summary  of  open  system 
maintenance  planning  activities  by  program  phase  is  provided  in  table  4-2. 

Another  important  maintenance  planning  concept  is  the  use  of  NDI/commercial  items  that  have  existing 
support  systems.  This  allows  the  leveraging  of  investments  already  made;  consequently,  the  use  of  existing 
NDI/commercial  item  support  mechanisms  is  less  expensive  than  developing  a  new  support  system.  The 
maintenance  planner  must  also  evaluate  each  OEM's  repair  and  supply  support  capabilities  before  making 
maintenance  planning  decisions. 


PHASE 

ACTIVITIES 

Phase  0 

Identify  candidate  standards. 

Concept  Exploration 

Identify  supportability  parameters  of  standards  linked  technologies. 

Determine  NDI/commercial  items  implemented  and  how  to  support  them. 
Analyze  maintenance  requirements  in  all  operating  environments: 

•  Research  and  Development  (R&D) 

•  Training 

•  In-Service  Engineering  Agent  and  Cognizant  Field  Activity 
(ISEA/CFA) 

•  Software  Support  Activity  (SSA) 

•  Integration 

•  Installation  and  checkout 

•  Fleet  operational 

Phase  I 

Refine  candidate  standards  selections. 

Program  Definition  and  Risk 

Refine  supportability  parameters  of  standards  linked  technologies. 

Reduction 

Determine  NDI/commercial  items  implemented  and  how  to  support  them. 
Analyze  maintenance  requirements  in  all  operating  environments: 

•  R&D 

•  Training 

•  ISEA/CFA 

•  SSA 

•  Integration 

•  Installation  and  checkout 

•  Fleet  operational 

Select  products. 

Choose  or  develop  the  associated  support  systems. 

Consider  product  and  system  level  Built-in-Test  (BIT)/Built-in-Test 

Equipment  (BITE )  functions. 

Consider  fault  tolerant  designs. 

Phase  II 

Select  and  integrate  products. 

Engineering  and 

Develop  the  associated  support  systems. 

Manufacturing  Development 

Develop  product  and  system  level  BIT/BITE  functions. 

(E&MD). 

Implement  fault  tolerant  designs. 

Plan  for  upgrade  and  retrofits. 

Phase  III 

Establish  fleet  support. 

Production, 

Plan  for  upgrade  and  retrofits. 

Fielding/Deployment,  and 
Operational  Support. 

Monitor  and  coordinate  for  effective  support. 

Table  4-2  Open  System  Maintenance  Planning  by  Program  Phase 
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4.2. 1  Hardware  Maintenance 

No  single  maintenance  concept  can  apply  to  all  open  system  interface  standards  based  hardware  items. 
However,  the  following  influences  should  be  examined. 


HARDWARE  MAINTENANCE  PLANNING  CONSIDERATIONS 

•S  Uniqueness  of  product. 

•  Is  DoD  the  only  user? 

•  On  the  platform  or  within  the  system? 

Spares  availability  from  other  places  within  the  system  or  from  less  important  systems  at  the  same  user 
location  (platform,  locality). 

System  design  provides  redundancy  or  graceful  degradation  to  acceptable  performance  levels. 

A  degraded  performance  level  sufficient  to  meet  system  and  higher  level  requirements  while  waiting  for 
supply  support  action  to  replace  failed  parts. 

BITE,  performance  monitoring,  and  other  automated  fault  detection  and  reporting  capability. 

•  Capability  and  ability  to  integrate  that  capability  with  other  assemblies,  both  developmental  and  NDI. 
•S  Uniqueness  and  availability  of  adequate  technical  documentation. 

Availability  of  spares  an/or  repair  action  from  commercial  or  organic  sources. 

■S  Availability  of  adequate  technical  consultation. 

•S  Design  stability  of  the  product. 

•  Rate  and  nature  of  changes. 

•S  Warrantee  services. 


In  general,  hardware  conforming  to  open  system  interface  standards  will  tend  to  contain  large  numbers  of 
NDI/commercial  items  at  the  replaceable  assembly  level.  If  the  assembly  is  NDI/commercial  item,  repair  services 
are  usually  available  through  commercial  facilities,  negating  the  need  and  economic  justification  for  creating 
organic  repair  capability.  Furthermore,  the  detailed  technical  information  needed  to  enable  repair  of  board  level 
NDI/commercial  item  products  is  often  unavailable  to  customers.  Stockage  of  board  level  spares  will  generally  cost 
far  less  than  the  combination  of  people,  training,  equipment,  and  piece  parts  needed  to  enable  repair  of  failed 
assemblies.  Also,  warranties  tend  to  be  voided  by  non-authorized  attempts  to  troubleshoot  or  repair  failed  items. 

Built  in  redundancy  or  designing  for  graceful  degradation  of  performance  may  be  used  to  compensate  for 
slowness  of  replenishment  time  for  parts  which  for  whatever  reason  cannot  be  made  available  as  ready  spares.  For 
example,  if  a  computer  contains  six  interchangeable  cards  of  which  only  four  are  needed  most  of  the  time,  graceful 
degradation  of  performance  to  a  known  level  may  be  acceptable  until  the  replenishment  can  be  accomplished. 

The  user  is  generally  dependent  upon  a  product  support  agent  like  the  ISEA,  system  program  office,  prime 
or  integration  contractor,  or  a  third  party  activity  (e.g.,  contractor,  mobile  technical  unit  (MOTU),  etc.)  for  supply 
support  and  processing  of  failed  items  into  and  through  the  repair  process.  Planning  and  implementing  this 
support  infrastructure  is  not  a  minor  task.  Providing  actual  repair  capability  at  the  user  location  may  be  even  more 
expensive,  and  less  practical. 

4.2. 1 . 1  Hardware  Diagnostics 

Diagnostics  may  present  a  challenge.  Diagnostic  requirements  need  to  be  resolved  prior  to  final  selection 
of  specific  standards.  Interoperability  of  the  inherent  fault  detection  and  localization  capability  needs  to  be 
established  as  part  of  the  derived  requirements.  Information  about  the  inherent  capabilities  provided  by  the 
interfaces  specified  in  the  standards  and  standard  profiles,  the  candidate  system  architectures,  and  the  specific 
products  being  considered  is  required..  Although  good  system  level  BIT  and  fault  localization  capability  may  be 
achievable  with  open  system  interface  standards  based  products,  not  all  implementations  may  be  capable  of  self 
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monitoring  and  error  message  transmission  in  a  manner  which  facilitates  convenient  maintenance  actions. 
Integration  of  products  from  different  designers  may  provide  acceptable  performance,  but  fault  detection, 
correction,  and  error  message  requirements  may  not  be  achievable. 

4.2. 1.2  Hardware  Maintenance  via  Upgrades 

Many  open  system  interface  standards  based  products  change  rapidly  in  response  to  market  forces  and 
technology  developments.  In  this  situation,  the  maintenance  concept  may  appropriately  include  support  via 
upgrades.  Upgrades  which  do  not  affect  interfaces  (form,  fit,  and  function)  may  often  be  easily  accommodated. 
Using  currently  available  items  may  be  more  cost  effective  than  executing  life-of-type  buys  or  other  methods  (e.g., 
repair)  by  which  availability  of  the  original  items  can  be  assured.  For  example,  one  would  not  make  a  life-of-type 
buy  or  purchase  depot  level  stock  for  30  megabyte  hard  drives.  Larger  capacity  drives  with  better  access  times  can 
be  conveniently  purchased  at  lower  costs  and  will  fit  into  the  same  slots  as  the  30  megabyte  drives  while  providing 
the  same  functionality  at  higher  performance  levels.  If  a  failed  item  is  no  longer  available,  then  a  newer  product 
may  have  to  be  used  as  a  replacement  item. 

Benefits  of  accomplishing  maintenance  via  upgrades  include  keeping  the  system  technically  up  to  date 
throughout  its  lifetime.  Costs  associated  with  this  maintenance  concept  include  resources  associated  with  ongoing 
market  survey  and  test  and  evaluation  (T&E)  activities,  as  well  as  a  need  for  close  coordination  between  the 
support  agent  (e.g.,  ISEA)  and  suppliers.  Tools  to  enable  use  of  this  concept,  such  as  a  parts  interchangeability 
matrix  and  organizational  role  changes,  are  discussed  elsewhere  in  this  Guide. 

This  maintenance  concept  requires  some  investments.  Many  of  these  investments  must  be  made  no  matter 
which  maintenance  planning  concept  is  used,  so  their  costs  cannot  reasonably  be  allocated  solely  to  maintenance. 
A  closely  knit  support  infrastructure  must  be  in  place  to: 

•  facilitate  technology  insertion. 

•  provide  the  managerial  capability  needed  to  provide  a  parts  interchangeability  matrix,  training,  and 

technical  data, 

•  ensure  that  new  items,  and  supporting  documentation  are  conveniently  available, 

•  provide  technical  expertise  to  assist  in  problem  resolution,  and 

•  coordinate  with  other  platforms  and  user  systems  to  enable  reuse  of  retired  assets. 

This  maintenance  concept  can,  and  usually  should  be  combined  with  conventional  repair  (usually  at  and 
by  the  OEM)  and  reuse  of  replaced  assets.  For  platforms  scheduled  for  retirement  within  five  years,  maintenance 
via  upgrade  is  generally  inappropriate.  Therefore,  replacement  parts  can  come  from: 

•  systems  being  upgraded, 

•  repair  of  failed  parts,  and 

•  other  user  systems  which  have  or  are  being  retired  or  upgraded. 

Attention  must  be  given  to  those  items  unique  to  the  DoD  as  opposed  to  items  created  and  supported  for 
the  benefit  of  other  user  markets.  Uniqueness  carries  with  it  a  series  of  economic  and  practical  disadvantages  that 
must  be  identified  and  mitigated. 

4.2.2  Software  Maintenance 

The  definition  of  software  maintenance  differs  from  that  of  hardware  maintenance  by  virtue  of  what  the 
configuration  item  comprises.  That  is,  hardware  configuration  items  (HWCI)  are  assemblies  of  electronic  and 
mechanical  components  that  have  different  physical  characteristics,  materials,  or  dimensions  which  are 
documented  via  drawings  and  specifications  to  define  HWCI  elements  and  characteristics.  In  contrast, 
configuration  item/computer  software  configuration  items  (CSCI)  are  a  compilation  or  an  assemblage  of  data 
streams  (bits/characters)  in  a  serial  arrangement  having  no  physical  attributes  other  than  the  medium  in  which  they 
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are  stored.  Hence,  software  maintenance  encompasses  those  actions  and  processes  required  for  subsequent 
upgrades  and  interim  fixes/releases  to  correct,  improve,  enhance,  or  add  functionality,  performance,  and  efficiency 
to  a  given  CSCI.  Software  maintenance  encompasses  three  separate  types  of  maintenance:  corrective  (correcting 
software  errors),  adaptive  (modifying  software  to  support  a  changed  environment  (hardware/software)),  and 
preventive  (implementing  enhancement  to  existing  code). 

In  general  software  maintenance  may  be  performed  by: 

•  an  organic  (government)  activity, 

•  non-organic  activity  (system  integrator,  or  product  vendor),  or 

•  both  depending  on  the  cost  and  availability  of 

•  required  data, 

•  tools, 

•  environment,  and 

•  skilled  personnel. 

The  availability  and  quality  of  data  (i.e.,  source  code,  design  documents,  data  flow,  etc.)  determines  the  amount 
and  type  of  maintenance  control  the  program  office  will  have  on  the  system. 

4.2.2. 1  Processes/Activities 

Software  maintenance  processes  and  activities  are  driven  by  the  surrounding  supporting  infrastructure  to 

address: 


•  problem  reporting  and  corrective  action  processes, 

•  software  configuration  management  (CM)  with  respect  to  the  system  CM  processes  (section  5.5), 

•  change  processes  (analysis,  development,  test,  and  release),  and 

•  implementation  processes  (build,  delivery,  and  installation). 

How  each  of  these  processes/activities  is  implemented  depends  on  the  nature  of  the  software  change  and 
who  performs  the  change.  For  example,  changes  to  commercial  software  items  should  only  be  implemented  by 
the  OEM.  The  OEM  specifies  the  costs  and  time  frame  for  interim  fixes/upgrades.  The  processes/activities  for 
developmental  software  are  defined  by  the  implementing  program.  Software  trouble  reports  are  developed  and 
corrections  funded  for  subsequent  change  implementation.  Whether  the  change  is  directly  from  the  vendor  or 
developed  by  the  government,  software  maintenance  modifications  should  be  fully  tested  prior  to  approval  for 
delivery  and  installation  on  fielded  systems.  The  government  does  not  necessarily  have  insight  into  the  quality 
parameters  and  process  in  which  the  commercial  item  modification/update  was  made.  The  time  frame  from 
proposed/planned  change  to  implementation  is  characteristically  quicker  than  in  the  past. 

Changes  that  involve  modification  only  to  the  application  program  interface  (API)  (whether  commercial 
item  or  development)  can  be  effected  by  the  organic  support  activity  providing  the  supporting  technical 
documentation  defining  the  interface  design,  support  equipment,  facilities,  and  personnel  resources  are  available. 

4.2.2. 2  Open  System  Impact  on  Software  Maintenance 

The  distinguishable  characteristic  of  open  systems  is  that  open  system  interface  standards  based  products 
support  hardware  independence.  The  recognized  benefits  of  open  system  interface  standards  based  products  with 
respect  to  software  maintenance  translate  from  modularity  and  commonality  of  usage  into  costs  savings  (in  less 
manpower/personnel,  facilities,  and  support  resources)  to  the  government  specific  to  adaptive  maintenance.  That 
is,  the  DoD  mandate  for  commercial  software  procurements  has  caused  a  shift  from  full  system  product 
development  as  well  as  lessened  proprietorship  among  OEMs  to  participation  in  the  ‘plug  and  play'  environment. 
As  a  result,  this  shift  in  product  development  has  promoted  adaptability  among  products  in  areas  of  modifiability, 
expandability,  portability,  flexibility,  and  interoperability.  This  product  adaptability  in  turn  supports:  greater  data 
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format  transparency  and  access  among  heterogeneous  platforms  translating  into  less  cost  and  less  risk  to 
government  while  promoting  faster  technology  insertion  into  the  fleet. 


SOFTWARE  MAINTENANCE  PLANNING  CONSIDERATIONS 

■S  Uniqueness  of  product. 

•  Is  DoD  the  only  user? 

•  On  the  platform  or  within  the  system? 

■S  Uniqueness,  availability,  usefulness  of  software  related  technical  documentation. 

■S  Availability  of  technical  consultation. 

■S  Design  stability  of  the  product. 

•  Rate,  and  nature  of  changes. 

■S  Software  upgrades  (i.e.,  cost,  tracking  changes,  anti-virus  and  tampering  features,  data  availability). 

■S  Software  license  factors  and  degree  of  coverage. 

■S  Length  of  product  support. 

■S  Escrow  account  agreements,  especially  the  cost  effectiveness  of  buying  the  feature. 

■S  Software  support  factors  including  cost-effectiveness  trade-off  studies. 

■S  BIT/BITE  capabilities. 

■S  CM  factors  (i.e.,  change  periodicity  being  experienced,  product  tracking  capabilities,  product  line  similarity, 
source  code  availability,  standards  profile  to  product  information,  conformance  testing  data,  etc.). 

■S  User  data/track  record/reliability  factors. 

•S  Warrantee  service. 


4.2. 2. 3  Approaches/Guidelines 

The  software  OSE  must  be  defined.  This  can  be  accomplished  by  specifying  requirements  that  support 
open  system  interface  standards  based  products  and  the  use  of  commercial  software  to  support  system 
requirements.  Software  maintenance  planning  should  be  focused  on  standard  APIs.  Specifying  software  support 
metrics  early  can  greatly  reduce  the  required  software  maintenance  costs  later.  Two  thirds17  of  software  life  cycle 
costs  are  expended  on  maintenance  support. 

In  the  case  of  commercial  items,  the  government  will  be  dependent  on  a  vendor  for  software  maintenance 
via  a  warranty  or  service  contract.  This  normally  suffices  to  meet  supportability  and  software  applicability 
concerns.  Alternatively,  the  program  office  may  sometimes  be  able  to  purchase  data  and  data  rights,  and  fund 
development  and  maintenance  of  a  SSA  to  perform  the  same  role.  Quite  often,  vendors  are  not  willing  to  sell  their 
data  rights;  therefore,  the  government  should  be  concerned  that  the  software  development  firm  may  go  out  of 
business.  One  method  to  reduce  risk  is  to  establish  an  escrow  account,  whereby  a  third  party  holds  custody  of  the 
source  code  and  supporting  technical  data  required. 


17  Chapter  7:  Post  Deployment  Software  Support,  Mission  Critical  Computer  Resources  Management  Guide  (US 
Government  Printing  Office,  Washington,  DC,  undated),  7-3. 


56 


31  December  1996 


NGCR  Document  No.  AST  003  ver  1.0 


The  following  SOW  wording  is  provided  for  escrow  account  establishment: 

Escrow  account  for  system  code.  The  contractor  shall  place  two  copies  of  the  source  code 
(on  magnetic  media,  formatted  to  be  read  and  compiled  on  contract  equipment  as  part  of 
this  contract)  in  an  escrow  account  under  the  custody  of  a  third  party  mutually  agreeable  to 
the  government  and  the  contractor.  Each  copy  of  source  code  shall  be  kept  current  by  the 
contractor  for  the  life  of  the  contract.  The  agreement  with  the  custodian  shall  include  a 
binding  provision  for  immediate  release  of  all  source  code  to  a  designated  government 
representative  upon  receipt  of  a  request  from  the  Contracting  Officer  which  states  that 
software  support  under  this  contract  has  failed  to  meet  the  terms  and  conditions  specified. 

In  the  event  that  more  than  one  baseline  is  being  maintained  (that  is,  for  the  purpose  of 
upgrading  the  system  software)  that  source  code  shall  also  follow  the  escrow  procedures 
stated  in  the  above  paragraph. 

If  the  software  product  is  developed  for  a  specific  DoD  program,  some  benefits  of  open  system  interface 
standards  (e.g..  Portable  Operating  System  Interface  (POSIX))  may  still  be  obtainable.  These  benefits  come  from 
the  stability  of  APIs  across  generations  of  operating  system  and  application  software  programs.  Benefits 
attributable  to  stable  APIs  are: 

•  reduced  quantity  of  new  source  lines  of  code  (SLOC)  needed,  and 

•  more  legacy  software  product(s)  will  be  reusable  (e.g.,  application  data  and  application  software). 

4.2. 2. 3.1  Software  Maintenance  Example 

As  a  means  to  demonstrate  the  ease  of  software  maintenance  through  the  use  of  open  system  interface 
standards,  this  example  (table  4-3)  discusses  the  reusability  of  standard  APIs  from  one  generation  of  application 
software  to  the  next.  In  our  example: 

•  The  host  hardware  and  operating  system  are  changed  concurrently. 

•  The  application  program  is  changed  to  provide: 

•  the  same,  similar,  or  somewhat  improved  functionality,  and 

•  the  same,  similar,  or  somewhat  improved  performance. 

•  48  of  the  60  APIs  are  the  same  for  the  old  and  new  software. 

•  Only  12  of  the  60  APIs  are  no  longer  used. 

•  Used  17  new  standard  APIs  for: 

•  performance  enhancement, 

•  streamlining  code,  and 

•  improved  technology. 


ORIGINAL 

REUSED 

NEW 

Application  software  program 

36  software  modules 

Replaced  5  software  modules 

60  standard  APIs 

48  standard  APIs 

17  standard  APIs 

Operating  system 

Moved  to  current  release 

Hardware:  chassis  and  7  lowest 
replaceable  units  (LRUs) 

Hardware:  chassis  and  6  LRUs 

Replaced  obsolete  LRU 

Table  4-3  Example  Of  Software  Upgrade  Transition  With  Relationship  To  APIs 
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Compared  to  a  traditional  scenario,  in  which  the  operating  system  was  part  of  the  application  software,  or 
existed  as  a  proprietary  (single  user)  “executive"  program,  this  update  scenario  takes  less  time  and  costs  less 
money.  Some  of  the  reasons  this  is  true  are: 

•  The  operating  system(s)  may  be  a  commercial  product  which  complies  with  open  system  interface 
standards: 

•  There  is  a  pool  of  users. 

•  The  costs  and  time  associated  with  development,  testing,  and  support  are  spread  across  the  pool  of 
users. 

•  Software  maintenance  costs  are  not  the  same  for  each  and  every  SLOC. 

•  The  costs  could  be  dependent  on  the  type  of  change. 

•  The  change  could  impact  other  computer  resources. 

•  Complexity,  time,  and  cost  of  changing  the  application  software  may  be  dramatically  reduced  because 
most  of  the  interfaces  to  the  operating  system  may: 

•  contain  many  of  what  would  otherwise  be  application  program  functions, 

•  remain  the  same  from  one  generation  of  operating  system  and  application  program  software  to  the 
next,  and 

•  preserve  usability  of  the  software  modules  (products)  which  implement  these  functions. 

•  Eighty  percent  (48/60)  of  the  APIs  in  this  example  continued  to  be  used  from  one  generation  of 
application  software  to  the  next. 

•  This  stability  of  interfaces  implies  that  most  software  modules  of  the  application  program  can  remain 
constant,  i.e.,  there  is  no  need  to  develop  new  code. 

•  The  old  code  can  be  recompiled  to  the  new  host  hardware,  essentially  reused,  saving  code  development 
and  test  costs,  especially  for  mission  critical  systems. 

4.2.2. 4  Related  Costs 

In  the  past  the  cost  of  software  maintenance  has  consumed  more  than  two  thirds  (2/3)  of  the  software  life 
cycle  costs.  However,  and  as  noted  earlier,  software  maintenance  costs  can  be  minimized  with  an  open  systems 
approach  through  the  use  of  standard  APIs  (stability)  to  effect  adaptive  changes.  Stability  of  APIs  translates  to  less 
risk  in  development,  modification,  testing,  schedule,  performance,  and  overall  software  maintenance  costs.  Most 
of  the  APIs  offered  by  commercial  applications  support  openness  (primarily  to  ensure  vendor  preservation  in  the 
marketplace). 

Some  software  maintenance  costs  to  consider  include: 

•  Make  (or  modify)  vs.  buy  costs  (that  is,  perform  a  make  vs.  buy  estimate  to  assess/analyze  the  buy  costs  of 
the  software  modification  suggested). 

•  Rehosting  existing  code  (e.g.,  gray  box  computer  language  (i.e.,  CMS2)  supporting  legacy  systems  vice 
new  development  to  emulate  existing  functionality). 

•  SLOC  required  to  affect  the  modification  and  its  impact  on  memory  reserves,  processing  speed,  response 
time(s),  etc.  which  may  translate  into  costs  beyond  that  of  SLOC  modifications. 

•  SLOC  complexity. 

•  Application  code  updates  resulting  from  periodic  updates  to  commercial  software. 

Even  if  all  or  part  of  the  software  program  is  produced  and  maintained  by  an  outside  supplier,  the  costs 
will  still  be  present.  Commonality  among  multiple  customers  of  the  open  system  interface  standard  software 
allows  the  supplier  to  divide  the  cost  of  making  changes  among  more  customers,  thus  lowering  the  effective  cost  to 
each  customer. 

Ligure  5-1,  Product  Survey  Lorm,  may  serve  as  a  basis  (or  starting  point)  for  comparison  of  alternatives 
and  budget  and  program  plan  development.  Although  the  form  was  developed  for  NDI/commercial  items  market 
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surveys  and  acquisition  planning,  the  charted  cost  accounts  can  serve  as  prompts  for  developmental  planning  and 
as  a  means  to  estimate  costs  of  a  proposed  software  change 

4.3  Manpower  and  Personnel 

As  the  technologies  of  communications  and  computers  become  more  common  and  open,  there  is  a 
decrease  in  the  number  of  different  information  sources  needed  for  support  of  a  variety  of  systems  that  use  these 
technologies.  Open  systems  promote  commonality  among  products,  whether  they  be  hardware,  operating  systems,  or 
applications.  This  implies:  reduced  number  of  Navy  Enlisted  Codes  (NEC),  reduced  number  of  operations  and 
maintenance  personnel,  and  portability  of  personnel  across  the  different  hardware/software  boundaries. 

Proprietary  systems  suggest  uniqueness,  which  in  turn  brings  the  need  to  retain  uniquely  qualified  support 
personnel.  The  commonality  of  the  open  system  interface  standards  based  products  (e.g.,  POSIX-based  operating 
systems)  eases  development  of  new  software  and  changes  to  old  software  by  the  SSA,  reducing  the  number  of  personnel  at 
the  ISEA/CFA  and  the  SSA. 

Use  of  open  system  interface  standard  based  products  in  the  fleet  reduces  the  number  of  unique  NECs  for 
systems  deployed.  A  decreased  number  of  unique  NECs  generalizes  the  training  requirements  and  develops  personnel 
who  have  a  broader  knowledge  base.  For  example,  tactical  computers  (TAC)  are  used  across  many  system  domains 
resulting  in  a  common  knowledge  base  (NEC  type)  across  these  systems.  The  information  affords  the  opportunity  for 
fewer  people  to  be  able  to  maintain  and  operate  more  different  systems.  Depending  on  actual  workload,  this  will  tend  to 
bring  about  an  opportunity  to  decrease  the  total  number  of  personnel  required  at  the  platform  level.  A  manpower 
requirements  analysis  must  be  conducted  to  optimize  the  mix  of  personnel  and  skills. 

Technology  can  provide  ways  to  increase  efficiency.  A  move  to  open  systems  will  enhance  this  by  creating  a 
personnel  portability  as  systems  may  be  designed  in  such  a  way  that  personnel  can  transfer  their  training  and  experience 
from  one  system  to  another. 


MANPOWER  AND  PERSONNEL  PLANNING  CONSIDERATIONS 

■S  Determination  of  maintainer  proficiency  levels  versus  the  projected  technology  and  product  insertion. 
■S  Identification  of  number  and  type  of  operator  and  maintainer  levels  to  meet  requirements. 

■S  Identification  of  deltas  between  legacy  and  “to  be”  architecture  needs. 

■S  Determine  if  existing  skill  levels  are  ample  to  meet  new  and  existing  requirements. 

■S  Degree  of  difficulty  of  personnel  training  to  operate  and  maintain  the  end-item. 

■S  Product/system  capability  to  provide  self  test  and  training  services. 

■S  Similarity  of  technology  with  current  platform  systems,  products,  and  services. 

■S  User  maintenance  instructions. 

■S  Operator  instructions. _ 


4.4  Supply  Support 

Supply  support18  within  the  Services  has  traditionally  been  stated  as  : 

•  Availability  of  organizational,  intermediate,  and  depot  level  repair  parts,  insurance  spares,  and 
replenishment  parts,  in  the  supply  system  to  replace  the  above  as  they  are  used. 

•  Supply  support  includes  technical  documentation  that  provides  the  maintenance  philosophy  for  the  end 
item  equipment  and  identifies  the  parts  required  to  support  the  maintenance  philosophy. 

Just  as  in  past  DoD  acquisitions,  open  system  acquisition  supply  support  also  needs  to  provide  the  spare  and  repair 
parts  necessary  to  operate  and  maintain  the  end  item  at  defined  levels  of  maintenance  that  meets  specified 


SPAWAR  P4000.14,  SPAWAR  Desk  Guide  for  Supply  Support  Planning  and  Execution. 
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operational  availability  requirement  usually  defined  in  the  ORD  and  other  acquisition  documentation.  Current 
DoD  policy19  requires  that  support  concepts  for  new  or  modified  acquisitions  maximize  contractor  long-term 
logistics  support  that  combines  depot-level  maintenance  and  wholesale  and  selected  retail  materiel  management 
functions  while  still  maintaining  limited  organic  core  depot  maintenance  capability  to  meet  essential  DoD  defense 
requirements.  Life  cycle  costs  and  associated  risk  factors  are  key  selection  criteria’s. 

In  most  new  or  modified  acquisitions,  systems  will  move  from  existing,  legacy  and  proprietary 
environments  toward  an  open  environment.  The  planning  for  supply  support  must  therefore  include  both  legacy 
and  open  systems.  The  shift  to  an  open  system  environment  provides  an  opportunity  for  more  rapid  technology 
insertion  and  upgrade.  Upgrade  can  proceed  on  a  constant  basis.  Technology  upgrade  and  supply  support 
functions  become  intermingled.  As  more  open  system  interface  standard  compliant  products  become  available, 
initial  acquisition  costs  may  be  reduced.  Since  the  products  are  not  necessarily  interchangeable,  supporting  a 
particular  baseline  with  functionally  equivalent  products  from  multiple  sources  incurs  risk.  The  Navy  should  retain 
the  right  to  procure  directly  from  the  OEM  (rather  than  the  system  prime  or  integration  contractor)  to  obtain  the  best 
value. 


Commercial  support  systems  (repair,  technical  support,  etc.)  that  exist  for  vendor  products  that  meet 
standards  applications  should  be  utilized  whenever  possible  rather  than  developing  a  redundant  capability.  Typical 
support  that  can  be  leveraged  includes  support  to: 

•  R&D  facilities  (usually  consisting  of  technical  consultation  and  documentation  of  interface  compliance 
and  features), 

•  integration  and  test  facilities  (usually  including  user-oriented  technical  documentation  as  well  as  repair 
support  and  technical  manuals  close  to  those  which  will  support  field  usage), 

•  user  location  (field)  installation  and  test  activities  (same  as  integration  facilities  but  with  more  and 
different  spare  parts  and  technical  support),  and 

•  long-term  operational  environments  (i.e.,  fleet  ships,  aircraft,  etc.). 

Product  choice  and  usage  decisions  must  consider  the  availability,  completeness,  and  quality  of  support  to 
all  these  environments. 

The  various  OSA  standards  themselves,  combined  with  vendor  support  of  products  that  implement  them, 
will  create  a  stable  board,  box,  and  system  level  development  environment  that  will  routinely  outpace  the  DoD 
unique  product  environment  by  up  to  50%.  This  estimate  is  based  on  actual  industry  planning  and  experience  in 
an  OSA  environment.  The  impacts  of  this  are  manifold: 

•  Product  support  will  be  required  much  sooner  than  in  past  experience.  We  must  be  ready  to  contract  for 
support  and  training  of  our  infrastructure  such  as  our  Warfare  Center  personnel,  if  they  are  to  be  involved 
in  oversight  of  product  development  or  fleet  users  support. 

•  Product  support  will  depend  on  a  variety  of  usage  environments  and  needs.  DoD  operational  system 
environments  are  but  a  small  and  perhaps  insignificant  part. 

•  We  must  plan  how  to  live  with  a  product  before  committing  to  using  it  in  our  systems. 

•  Program  planning  and  budgeting  must  reflect  these  new  realities. 

In  most  cases,  commercial  replenishment  of  spares  can  be  anticipated  for  products  already  used  and 
supported  in  commercial  environments.  The  mechanics  for  routine  processing  of  requisitions  on  the  commercial 
market  through  DoD  Inventory  Control  Points  (ICP)  already  exist.  By  using  electronic  data  interchange  (EDI),  a 
user  should  save  time  and  money. 

4.4.1  Supply  Support  Planning 


19  DoD  5000.2-R,  15  March  1995,  Part  3,  page  14. 
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As  systems  and  products  become  more  technologically  advanced  and  complex,  the  need  for  planning  is 
even  greater.  The  need  to  plan  earlier  is  critical  in  an  open  system  environment.  Choice  of  initial  standards  and 
products  influences  the  ability  to  support  the  product  in  the  future,  and  hence  useful  life.  The  costs  of  system 
support  are  largely  influenced  by  decisions  made  in  the  early  phases  of  the  life  cycle.  Supply  support  planning  and 
sparing  must  support  the  maintenance  decisions.  In  an  open  system  environment  the  need  to  achieve  the  required 
A0  is  still  present.  Performance  must  be  maintained. 

The  supply  support  process,  as  it  applies  to  an  open  systems  acquisition  effort,  must  be  capable  of  reacting 
to  rapid  or  frequent  changes  in  demand,  vendor  access  or  availability,  cost  factors  of  selected  end  items,  technology 
enhancements,  engineering  judgments  resulting  from  analyses  or  reliability  data  collected  over  time,  and  other 
factors  such  as  the  end  of  all  production.  Support  concepts  vary  from  program  to  program.  For  example,  in  one 
case  failed  end  items  or  LRU's  are  sent  to  an  established  deport  for  analysis,  repair,  or  modification  until  design  or 
support  decisions  are  stabilized.  In  another  case  the  support  concept  might  be  to  lease  support  from  either  the 
prime  integrator  for  a  period  of  time  to  establish  useful  reliability  and  support  data.  The  key  is  that  when  dealing 
with  an  open  systems  end  item,  unique  supply  support  issues  may  influence  the  decisions  on  how  to  provide  and 
manage  the  support  to  the  end  item  user. 

Provided  below  are  some  supply  support  considerations  that  should  be  planned  as  part  of  an  open  system 
acquisition. 


SUPPLY  SUPPORT  PLANNING  CONSIDERATIONS 

■S  Identification  of  maintenance  echelons  required  to  meet  user  needs. 

■S  Support  environment  (weather,  shelter/non-shelter,  etc.)  considerations. 

■S  Warranty  procedures  and  cost  effectiveness  over  life  cycle  of  end-item. 

■S  Organic  support  depot  cost-effectiveness. 

■S  Commercial  depot  support  capabilities. 

•  On-site  capabilities. 

•  Turn-around  times. 

•  Lead  time. 

■S  Single  point  service  restrictions  of  end-item. 

■S  Spare  part  lead  times. 

■S  Availability  of  tools,  test  equipment,  computer  support  resources,  calibration,  technical  manuals  (etc.). 

■S  Manufacturers  commitment  for  out-year  parts  support. 

■S  Repair  parts  availability  and  support  production  line  continuance. 

■S  On-board  spare  requirements. 

■S  Availability  of  depot  level  support  and  spares  vs.  Reliability  and  cost-effectiveness  issues. 

■S  Determination  of  end  item  replacement  concept  (repair  or  discard  item). 

■S  Determination  if  historical  data  of  end  item  correlates  directly  with  item  being  considered  for  integration  or 
sparring. 

■S  Cost-effectiveness  of  establishing  manufacture  support  for  end  item(s)  versus  Service  depot  support. 

■S  Joint  programs/products  supply  support  factors  for  similar  or  identical  end  item(s). 

■S  Navy  Inventory  Control  Point  (NAVICP)  support  of  the  same  end  item  and  established  length  of  the  support 
in  place. _ 


4.4.2  Selecting  Sources 

Open  systems  provide  new  dimensions  to  supply  support  that  are  not  always  apparent  in  a  proprietary  system. 
Open  systems  allow  optimal  product  choices  to  be  made,  based  on  interface  and  performance  attributes  from  a  wider  range 
of  sources.  This  freedom  of  choice  is  not  without  risks.  Assuming  the  products  are  equal  (interchangeable)  in  form,  fit, 
and  function,  a  choice  must  be  made  in  both  product  and  product  source. 
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Most  implementations  will  involve  a  combination  of  legacy  or  proprietary  systems  along  with  open  system 
interface  standards  based  products.  The  market  survey20  should  be  used  as  a  basis  for  identifying  candidate  products  and 
sources.  In  selecting  open  system  interface  standard  based  products  the  user  must  be  concerned  with  the  OEM's 
experience  with  proprietary  systems  and  open  systems,  both  hardware  and  software.  This  information  forms  the  basis  for 
solutions  to  integrating  the  old  with  the  new.  No  matter  the  source  or  product  selected,  it  will  have  to  be  qualified  for  use 
in  the  existing  system. 

Interface  commonality  allows  the  use  of  different,  yet  interchangeable,  replacement  items  (i.e.,  sparing 
with  a  new  product  rather  than  a  new  copy  of  the  old  product).  The  supply  support  team  must  be  familiar  with 
appropriate  interface  documents  to  identify  candidate  items  for  sparing.  The  goal  is  to  have  interchangeable  spares 
alternatives  that  meet  performance  and  interface  requirements,  or  have  a  work-around  in  place  to  avoid 
performance  degradation. 

hi  obsolete  product  cases  where  interchangeable  products  are  not  available,  functionally  equivalent  products 
shorten  the  product  replacement  time.  For  example,  the  functions  are  the  same,  but  the  partitioning  is  different.  This  will 
require  some  software  changes  to  enable  use  of  the  new  product. 

Some  processes  can  make  the  change  easier  to  handle.  One  is  the  parts  interchangeability  matrix  (PIM),  used 
to  document  applicability  of  new  and  old  items  to  furnish  support  of  product  baselines  being  supported.  The  PIM 
(section  5.5.9,  table  5-1)  documents  which  parts  can  be  used  with  each  other.  It  covers  the  interchangeability  of 
hardware  and  software  elements  (generations  of  the  part  or  part  family)  across  the  different  system  and  unit  level 
baselines. 

Another  process  is  reutilization  of  replaced  system  assets,  including  upgraded  platforms.  Basically,  the 
process  consists  of  reusing  hardware  that  has  been  replaced  by  more  current  baselines  to  support  platforms  that 
have  not  yet  been  upgraded.  A  process  already  in  use  but  receiving  little  publicity,  is  reuse  of  residual  assets  from 
decommissioned  ships  and  aircraft. 

4.4.3  Supply  Support  Data 

The  requirements  for  provisioning  technical  documentation  (PTD)  have  significantly  changed.  The 
minimum  amount  of  PTD  required  for  commercial  item  is  price,  part  number,  and  reliability  data  (preferred  but 
not  necessarily  required).  In  this  situation  the  government  develops  the  additional  provisioning  data.  In  addition 
to  the  greatly  increased  ability  to  handle  and  exchange  data  inherent  with  today’s  computerized  business  systems, 
the  use  of  open  systems  based  commercial  item  often  eliminates  the  need  for  traditional  PTD  delivery.  The  rapidly 
moving  commercial  base  implies  shorter  life  cycles  for  product  availability.  The  rate  of  change  inherent  in 
commercial  products  affects  how  frequently  new  PTD  data  is  generated.  Product  obsolescence  occurs  sooner, 
resulting  in  a  life  cycle  problem  for  buying  spares  and  support.  When  the  costs  associated  with  procurement  of 
PTD  are  deemed  excessive,  then  provisioning  alternatives  should  be  sought.  If  the  integration  contractor  can  not 
provide  PTD,  the  ISEA/CFA  and/or  support  contractor  may  be  a  good  choice  to  do  so. 

For  handling  supply  support  and  PTD,  the  NAVICP,  in  conjunction  with  Navy  Supply  Systems  Command 
(NAVSUP),  has  developed  an  innovative  method  referred  to  as  “just  in  time”  supply  support.  This  method  uses  the 
commercial  support  infrastructure  instead  of  the  military  supply  support  system.  The  integrator  or  OEM  provides 
user  manual  type  information  and  also  develops  the  allowance  parts  list  (APE)  using  only  selected  data  elements. 
The  APL  is  available  at  a  fraction  of  the  time  and  cost  of  traditional  PTD  methods.  This  APL  appears  the  same  to 
the  fleet  user  and  allows  him  to  identify  and  requisition  material  using  standard  military  requisitioning  and  issue 
procedures  (MILSTRIP).  However,  once  the  MILSTRIP  requisition  is  submitted,  the  inventory  control  point  uses 
electronic  purchase  orders  transmitted  directly  to  the  vendor  in  commercially  developed  and  recognized  formats. 
The  vendor  then  ships  the  spare  part  from  the  commercial  warehouse  directly  to  the  end  user.  The  government  is 
thereby  saved  the  direct  cost  of  storage,  as  well  as  the  distinct  risk  of  stocking  obsolete  spare  parts. 


20  The  market  survey  is  discussed  in  section  5.2. 
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4.4.4  Testing  of  New  or  Replacement  Spares 

Assurances  are  needed  that  the  products  and  supporting  spare  parts  procured  are  compatible  and 
interchangeable  with  existing  systems.  For  most  procurements,  it  is  not  enough  for  manufacturers  to  state  that 
their  products  are  compliant  with  the  required  standards.  Program  offices  may  want  to  perform  independent 
verification  of  the  claimed  conformance  as  a  matter  of  course.  Independent  testing  of  candidate  products  may  be 
required  to  ensure  compatibility  with  existing  systems.  Two  issues  must  be  considered: 

•  Does  the  product  conform  to  the  stated  interface  standard  or  profile?  If  the  unit  or  system  conforms,  then 
it  is  important  to  keep  that  conformance.  Otherwise,  future  upgrades  or  failure  induced  replacement 
efforts  may  be  jeopardized. 

•  Is  the  new  item  really  interoperable  with  the  host  unit  or  system?  Factors  other  than  the  interface 
standard,  such  as  processor  type  and  speed,  may  enhance  or  detract  from  host  unit  or  system  performance, 
or  make  the  new  item  unable  to  interoperate  successfully.  Risk  mitigation  is  obtainable  through  testing  of 
all  candidate  items. 

Conformance  testing  is  the  process  of  verifying  that  an  implementation  performs  in  accordance  with  a 
particular  standard,  thus  minimizing  risk.  Independent  conformance  and  interoperability  testing  should  be 
performed  prior  to  the  procurement  of  any  spares.  Consequently,  potential  conflicts  with  legacy  systems  are 
evaluated  and  made  known  prior  to  final  procurement  decisions. 

One  of  the  problems  with  standards  is  that  in  addition  to  the  minimum  (core)  requirements  needed  for 
compliance,  standards  contain  many  optional  features  that  manufacturers  may  choose  to  implement.  The  optional 
features  include: 

•  alternative  ways  of  implementing  a  feature, 

•  features  which  are  not  mandatory,  and 

•  ranges  for  values  of  variables. 

The  selection  and  interpretation  of  optional  features,  as  well  as  extensions  to  the  standards,  is  critical  to  the 
interoperability  and  compatibility  of  existing  systems  with  new  systems.  Unless  the  product  developer  has  been 
given  strict  guidance  as  to  allowable  or  disallowable  optional  features,  program  offices  will  need  to  be  concerned 
with  the  implementation  of  options  in  open  system  interface  standards. 

Independent  testing  of  candidate  products  may  be  required  to  ensure  compatibility  with  existing  legacy 
systems.  This  is  especially  true  for  the  procurement  of  spares.  Spares  procured  concurrently  with  the  production 
item  and  from  the  same  source  as  the  production  item,  are  likely  to: 

•  have  identical  configurations, 

•  be  interchangeable,  and 

•  be  tested  in  the  exact  same  manner  as  the  production  item  to  verify  interchangeability. 

Spares  procured  from  a  source  other  than  from  the  original  manufacturer  or  procured  from  the  same  producer  but 
later  in  the  products  life  cycle  should  be  tested  to  ensure  interchangeability  and  compatibility.  Changes  in  the 
product  resulting  from  upgrades  may  result  in  a  loss  of  interoperabitlity  or  reduced  functionality  when  used  in  a 
fielded  system(s). 

To  properly  compare  two  systems,  they  must  be  subjected  to  the  same  tests.  One  way  of  determining  the 
likelihood  of  two  systems  interworking  (being  interoperable)  with  one  another,  is  to  subject  both  systems  to  “live” 
testing  in  the  planned  system  configuration. 

As  open  system  interface  standards  are  developed,  conformance  testing  must  also  be  developed. 
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The  program  office  must  be  prepared  to  understand  the  test  results.  A  test  suite  for  a  given  protocol  is 
going  to  contain  a  number  of  discrete  tests.  For  each  test,  there  will  be  an  outcome  to  which  a  test  verdict  will  be 
assigned  (pass,  fail,  or  inconclusive).  The  program  office  will  need  to  define  the  pass/fail  criteria.  Clearly,  the 
requirements  need  to  be  precise  and  unambiguous  to  minimize  risk. 

As  more  and  more  suppliers  and  developers  base  their  product  lines  on  open  system  interface  standards, 
sources  for  standards-compliant  products  will  increase.  As  the  number  of  sources  increases,  several  things  will 
happen.  The  marketplace  will  drive  out  or  minimize  the  uniqueness  of  standards  implementations,  or  there  will  be 
more  uniqueness  in  order  to  capture  a  certain  niche  in  the  marketplace.  In  either  case,  the  risk  of  non¬ 
interchangeability  and  non-compatibility  still  prevails.  Caveat  emptor,  i.e.,  let  the  buyer  beware. 

4.5  Support  Equipment 

Commercial  open  system  interface  standards  are  developed  with  a  consideration  on  how  conforming 
products  will  be  maintained  in  commercial  markets.  Support  equipment  suitability  and  costs  are  primarily  affected 
by  technology  insertion  and  rate  of  product  changes. 

Currently,  fielded  systems  have  been  designed  to  use  test  equipment  normally  found  on  naval  platforms 
and  at  shore  sites.  Support  equipment  usually  requires  some  form  of  calibration  to  ensure  it’s  functionality.  The 
rapidly  expanding  use  of  commercial  items  that  are  not  driven  by  either  current  support  equipment  directives  or 
policies  requires  evaluation  of  proposed  commercial  items  (hardware  or  software)  support  equipment  requirements. 
Provided  below  are  typical  planning  considerations  for  identifying  and  providing  the  necessary  support  equipment 
to  the  end  user. 


SUPPORT  EQUIPMENT  PLANNING  CONSIDERATIONS 

C  BIT/BITE  capability  integrated  into  end  item. 

C  BIT/BITE  effectiveness  to  isolate  failures. 

C  Need  for  special  test  equipment  vs.  usability  of  current  on-hand  test  equipment. 

C  Unique  support  and  test  equipment  necessary  and  cost-effective. 

C  Need  and  cost  effectiveness  of  related  test  program  set  (TPS)  products  for  the  end  item. 

C  Calibration  standards  requirements/necessities. 

C  BIT/BITE  integration  factors  (physical  and  software  related)  and  costs. 

C  BIT/BITE  contractual  factors. 

■C  Support  equipment  operator  and  maintenance  training  requirements  and  costs. 

C  Does  the  rate  of  product  change  affect  TPSs. 

■C  Unique  support  equipment  use  on  other  pending  systems/equipment’s  (sharing  of  costs). 


Commonality  gained  through  increased  use  of  open  system  interface  standards  can  reduce  the  total 
number  and  different  types  of  test  equipment  required  resulting  in  reduced  total  costs.  Corresponding  test 
equipment  support  costs  can  also  be  reduced.  When  development  of  hardware  support  equipment  is  needed,  it  can 
be  accomplished  using  open  system  interface  standards.  When  TPSs  are  necessary  to  meet  system  support 
requirements,  the  change  rate  of  products  directly  affects  how  often  TPSs  will  need  to  be  changed. 

Depot  level  support  equipment  requirements  will  be  determined  on  a  case-by-case  basis.  The 
manufacturers  of  compliant  products  test  them  during  production.  These  same  tests  can  be  used  during  the  repair 
process  by  the  OEM.  There  may  be  no  need  for  Navy  investment  in  depot  level  support  and  test  equipment. 
However,  these  OEM  capabilities  may  be  insufficient  in  some  cases  to  meet  system  support  requirements.  Those 
cases  require  a  traditional  organic  depot  or  life-of-type  buy. 

The  number  of  unique  support  equipment  can  be  reduced  or  eliminated  when  LRU  and  system  level  BIT  is 
capable  of  isolating  failures  to  a  single  or  small  group  of  LRUs.  System  level  BIT  must  be  considered,  when 
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integrating  a  system,  in  the  partitioning  of  the  system  functions  and  integrated  across  all  functions  of  the  system. 
Many  open  system  interface  standards  provide  requirements  for  BIT  and  panel  indicators  that  visually  identify  the 
failed  LRU.  Sufficient  BIT  capability  can  mitigate  the  need  for  support  equipment  suites.  For  example,  system 
level  BIT  on  a  multi-unit  system  can  be  implemented  by  connecting  alternate  access  ports  (serial)  on  each  unit  to  a 
common  BIT  processor  that  runs  system  level  performance  monitoring  and  diagnostics.  BIT  capability  is  easier  to 
obtain  because  of  functional  partitioning  characteristics  inherent  to  open  system  interface  standards  usage. 

The  OEM  has  developed  production  line  testing  that  can  also  be  used  to  repair  a  failed  LRU.  When  LRU 
level  BIT  or  system  level  BIT  is  used,  the  number  of  unique  TPSs  can  be  reduced  by  returning  failed  items  to  the 
OEM  for  repair.  This  results  in  leveraging  the  OEMs'  development  of  production  and  acceptance  test  capabilities 
instead  of  developing  TPSs. 

4.6  Technical  Data 

Traditionally,  technical  data  began  with  the  system  specification  and  proceeded  forward.  In  an  open  system 
interface  standards  based  acquisition,  configuration  management  of  program  technical  documentation  and  system 
architecture  needs  to  start  during  concept  evaluation.  The  architectural  framework,  baseline  characterization,  and  target 
architecture  documents  along  with  data  required  for  supportability  are  all  necessary  technical  data.  All  contracts  are 
required  by  DoD  5000.2-R  to  require  on-line  access  to,  or  delivery  of,  their  programmatic  and  technical  data  in  digital 
form.  A  further  requirement  is  for  preferences  to  be  given  to  on-line  access  to  contractor  developed  data  through 
contractor  information  services  rather  than  data  delivery.  The  ability  to  rapidly  accrue,  compare,  and  dispense  technical 
data  accurately  can  mitigate  risks,  schedules,  and  cost  factors. 

The  use  of  open  system  interface  standards  relates  to  technical  data  in  two  basic  ways: 

•  Provides  a  definition  of  certain  interfaces  in  a  system. 

•  Contributes  to  the  increased  use  of  NDI/commercial  item  hardware  and  software  products  to  meet  system 

requirements. 

Open  system  interface  standards  can  be  viewed  as  technical  data  describing  significant  aspects  of  system  and 
product  design.  The  open  system  interface  standards,  in  conjunction  with  the  selected  profiles,  fully  define  the  interface 
requirements  for  a  product.  The  standards  and  standard  profiles  are  stable  entities  while  the  product  profile  can  change. 
Thus,  it  is  important  to  document  the  product  profile  for  future  reference  (e.g.  spares  procurement).  Open  system  interface 
standards  are  available  through  the  organizations  (e.g..  Institute  of  Electrical  and  Electronics  Engineers  (IEEE)  and 
American  National  Standards  Institute  (ANSI))  which  sponsor  development,  manage  configurations,  and  control 
updates.  Because  open  system  interface  standards,  standard  profiles,  and  the  product  profile(s)  define  the  interface 
implementations,  the  only  other  data  that  may  be  required  to  fully  document  the  interface  configuration  of  a  system  using 
NDI/commercial  items  is  the  commercial  performance  data  sheets  which  are  generally  widely  available  at  no  cost. 

Other  technical  data  issues  arise  from  open  system  interface  standards  contributing  to  the  use  of  NDI/commercial 
items  in  implementation  of  a  systems  architecture.  These  include: 

•  data  rights  versus  data  ownership, 

•  adequacy  of  drawings, 

•  adequacy  of  technical  manuals, 

•  modifications  to  hardware  or  software,  and 

•  support  after  obsolescence. 

The  adequacy  of  the  technical  data  for  NDI/commercial  items  should  always  be  examined.  The  use  of 
NDI/commercial  items  provides  an  environment  in  which  technical  data  may  be  inexpensive  and  accessible;  however,  the 
data  may  or  may  not  be  adequate.  Technical  manuals,  user's  manuals,  drawings,  and  training  materials  will  all  need  to  be 
examined  for  adequacy  prior  to  contract  and  procurement.  Data  adequacy  may  need  to  become  a  source  selection  criteria. 
If  a  product  with  inadequate  technical  data  is  to  be  selected,  licensing  arrangements  will  need  to  be  pursued  to  procure  the 
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required  data  either  from  the  OEM  or  another  source.  Normally,  the  government  requires  less  data  on  NDI/commercial 
item  products  than  on  developmental  products  so  acquire  only  the  data  that  will  actually  be  needed.  For  hardware, 
lifetime  buys  of  spares  can  mitigate  some  technical  data  requirements.  Another  possibility  might  be  a  license  to  use  the 
OEMs  drawings  if  the  OEM  goes  out  of  business.  For  software,  a  restricted  use  license  for  use  of  obsolete  source  code 
may  be  arranged. 

Many  commercial  standards  represent  the  least  common  agreed  upon  set  of  functionality.  This  implies 
that  functionality  (extensions)  must  be  added  to  ensure  a  “sensible”  implementation  results.  Beyond  the  standard, 
the  technical  data  must  also  include  documentation  of  all  such  extensions  and  modifications  to  the  given  standard. 
This  includes  vendor  provided  operating  manuals. 

Accurate  documentation  of  a  cabling  system  is  important  and  should  be  readily  available.  Technical 
resources  include  installation  guides  for  all  network  hardware  and  software  components.  They  sometimes  lack  the 
technical  details  of  network  operation,  however.  For  that  reason,  it  may  be  useful  to  obtain  technical 
specifications.  Every  network  interface  card  contains  controller  chips  that  implement  the  access  protocol;  the 
manufacturers  of  these  protocol  handler  integrated  circuits  can  provide  useful  hardware  specifics.  Software 
developers  also  provide  technical  documentation  on  their  protocols. 


TECHNICAL  DATA  PLANNING  CONSIDERATIONS 

■S  What  baseline  documentation  is  available? 

•  Drawing  materials  (i.e.,  level  2  or  3  drawings,  computer-aided  design/computer-aided  manufacturing 
(CAD/CAM)  data. 

•  Software  data  (i.e.,  software  requirements  specifications,  source  data/listings,  etc.). 

•  Historical  data  of  end  item  (reliability  predictions/reports,  trouble  reports,  engineering  change  proposals 
(ECP)/specification  change  notices  (SCN),  software  database,  etc.). 

•  Design  analysis  data  (i.e.,  finite  element  analysis  (FEA),  boundary  element  analysis  (BEA),  solid 
modeling,  interface  modeling  and  simulation). 

■S  Is  an  interface  standard(s)  profile  conformance  management  policy  in  place? 

•  Projected  or  selected  profile(s)  promulgated. 

•  IPT/program  managers  familiar  with  profile(s)  use  and  optimization. 

•  Tracking  mechanism  in  place  for  non-conformance. 

•  Tracking  of  emerging  standards/profiles  and  associated  training. 

•  Conformance  test  process/verification  in  place. 

■S  Is  a  method  for  verifying  conformance  in  place? 

•  Labs  capable  of  validating  product  lists/claims/performance. 

•  Alternative  testing  available. 

•  Assessments  of  available  test  data. 

•  Demonstrations  (i.e.,  portability,  interoperability,  and  vendor  independence  demonstrations). 

■S  What  are  the  technical  data  exchange  capabilities,  (i.e.,  access  of  various  support  data  via  electronic/digital 
form,  section  4.6.2  and  tables  4-4  and  4-5)  between  the  government  (including  NAVICP),  prime  contractor(s), 
subcontractors,  vendors,  and  suppliers? _ 


4.6.1  Technical  Data  Rights 

The  Defense  Federal  Acquisition  Regulation  (DFAR)  data  rights  supplement  (29  September  1995)  further 
defines  data  rights  protection  for  vendors.  The  DFAR  prohibits  the  government  from  requiring  vendors  to  supply  any 
technical  information  about  a  commercial  product  that  is  not  generally  provided  to  the  public.  The  government  has 
unlimited  rights  to  technology  that  is  developed  exclusively  with  government  funds  or  is  otherwise  available  without 
license  restrictions,  including  the  right  to  disclose  it  to  other  parties.  The  government  has  rights  to  use  technology 
developed  joindy,  with  federal  and  private  funds,  for  "government  purposes"  only  for  a  specified  period.  After  this  vendor 
protection  period  expires,  the  government  could  use  the  technology  as  it  sees  fit,  including  making  it  available  for 
commercial  use.  The  government  has  restricted  rights  to  technology  developed  at  private  expense,  based  solely  on  the 
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licensing  agreement  negotiated  with  the  vendor.  In  all  cases,  subcontractors  will  be  entitled  to  the  same  protection  as 
prime  contractors  under  the  regulations. 

4.6.2  Technical  Data  Exchange 

The  program  office  needs  access  to  databases  within  DoD  and  industry  to  conduct  an  adequate  market 
survey  upon  which  to  base  tradeoffs.  Market  survey  data  (e.g.,  product  descriptions  and  open  interface  standards 
and  tutorials)  can  constitute  the  initial  technical  data.  There  are  also  many  public  pathways  to  technical  data 
which  facilitates  ongoing  market  survey  and  program  communication.  The  Internet  provides  a  virtually  free 
network  of  world  wide  web  (WWW)  home  pages,  many  of  which  contain  technical  information  provided  by 
developers  and  producers  of  high  tech  electronic  products.  Standards  bodies  have  home  pages  too,  as  do  many 
computer  magazines  and  trade  associations.  A  wealth  of  technical  information,  as  well  as  points-of-contact,  can  be 
obtained. 

The  program  office  can  initiate  and  implement  mechanisms  to  facilitate  access  to  and  transfer  of 
programmatic  and  technical  data.  Technical  data  that  needs  to  be  exchanged  includes: 

•  program  management, 

•  engineering, 

•  reliability,  maintainability,  and  availability, 

•  design, 

•  prototype  development, 

•  manufacturing 

•  CAD/CAM, 

•  test, 

•  quality  assurance, 

•  configuration  management, 

•  technical  manuals  (standard  and/or  interactive  electronic) 

•  training  support, 

•  installation  control  drawings  and  installation  drawings, 

•  manufacturing/production  schedules, 

•  parts  data  from  many  sources,  and 

•  direct  insight  of  product  changes  and  market  trends. 

Many  commercial  manufacturers  are  implementing  their  own  technical  data  continuous  acquisition  and 
life  support  (CALS)  oriented  architecture,  often  using  products  that  comply  with  the  Joint  continuous  acquisition 
and  life  support  (JCALS)  standards,  infrastructure,  and  APIs.  Some  of  the  support  which  the  open  system  interface 
standards  based  JCALS  structure  can  provide  to  the  program  office  are  shown  in  table  4-4.  Table  4-5  describes 
applicable  CALS  standards  which  ensure  open  digital  data  exchange  capabilities. 


FEATURES 

BENEFITS 

UNIX/POSIX  Based. 

Promotes  portability  of  applications. 

X-Windows/MOTIF  GUI. 

Provides  consistent  look  and  feel;  user  friendly. 

Supports  Transmission  Control  Protocol/Intemet  Protocol 
(TCP/IP)  and  General  Open  System  Interconnection  Protocol 
(GOSIP). 

Enables  connectivity  to  most  Local  and  Wide  Area  Network 
(LAN/WAN)  environments. 

Support  for  personnel  computers,  X-Terminals. 

Protects  governments  capital  investment. 

UNIX  workstations,  MACINTOSHs,  and  communication 
environments  NDI/commercial  item  independence. 

Eases  technology  insertion;  not  dependent  on  specific 
vendors;  scaleable;  expandable;  protects  investment  in 
software;  reduces  development  costs. 

Supports  CALS  industry  standards. 

Supports  government/industry  digital  interoperability;  eases 
migration. 
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Table  4-4  JCALS  Open  System  Environment 


DoD 

INDUSTRY 

APPLICATION 

MIL-STD-1840 

Automated  interchange  of  technical  information 

MIL-D-28000 

IGES  (ANSI  Y14.26M) 

Vector  graphics;  CAD/CAM 

MIL-M-28001 

SGML  (ISO  8879) 

Text;  Documents 

MIL-R-28002 

CCITT  GR4  (ISO  8613) 

Raster  graphics;  Scanned  images 

MIL-D-28003 

CGM  (ISO  8623) 

2-D  vector  graphics  (technical  illustrations) 

MIL-STD- 1388-2 

Logistic  Support  Analysis  Record 

PDES/STEP  (ISO  10303) 

Product  Description 

EC/EDI  (ANSI  X.12/X.400) 

Electronic  exchange  of  business  information 

ACRONYMS 

CCITT  -  International  Telegraph  &  Telephone  Consultative  Committee  group  4  (ISO  8613) 

CGM  -  Computer  Graphics  Metafile  (ISO  8623)  EC/EDI  -  Electronic  Commerce,  Electronic  Data  Interchange  (ANSI  X.  12/X.400) 

IGES  -  Initial  Graphics  Exchange  Specification  (ANSI  Y14.26M)  PDES  -  Product  Data  Exchange  using  STEP 

SGML  -  Standard  Generalized  Mark-up  Language  (ISO  8879)  STEP  -  Standard  for  the  Exchange  of  Product  Model  Data  (ISO  10303) 

Table  4-5  Digital  Data  Exchange  Capabilities 


4.7  Training  and  Training  Support 

Training  and  training  support  can  be  achieved  more  efficiently  using  open  system  design  approaches.  Using 
open  system  approaches  in  conjunction  with  commercial  support  and  rapid  upgrades  requires  that  the  training 
program  be  reviewed  more  often  and  more  thoroughly  to  ensure  that  it  is  current  with  the  advancing  technology. 
As  open  system  interface  standards  usage  increases,  the  number  of  different  interface  types  decreases,  thereby 
reducing  the  required  number  of  NECs.  When  this  is  applicable  across  various  systems  and  platforms,  it  provides 
the  program  office  an  opportunity  to  leverage  the  existing  NECs  and  their  training  programs  for  support  of  a  new 
system.  The  use  of  NDI  provides  an  additional  opportunity  for  the  program  office  to  leverage  existing  training 
programs. 

CM  of  the  training  courses  and  materials  can  be  a  serious  challenge  in  an  open  system  environment.  The 
rate  of  change  in  the  products  being  supported  requires  planning  for  a  larger  than  traditional  CM  role  for  training 
courses  and  materials.  CM  is  discussed  elsewhere  in  this  Guide. 

The  full  range  of  training  needs  includes  the  developing  organizations  (i.e.,  program  office/warfare  center 
acquisition  team),  fleet  operators  and  maintainers,  the  in-service  engineering  activity,  training  instructors,  the 
SSA,  and  maintenance  personnel. 


TRAINING  AND  TRAINING  SUPPORT  PLANNING  CONSIDERATIONS 

■S  Is  the  OSE  related  technology  addressed  within  the  training  curriculum? 

■S  Can  the  training  be  incorporated  into  an  already  established  NEC? 

■S  Is  computer-based  training  feasible  for  meeting  the  system  training  requirements? 

■S  Can  the  system  provide  OJT  as  part  of  on-line  system  interaction  with  the  operator  and  maintainer? 

■S  Is  a  training  plan  in  place  to  provide  OSE  and  technology  training  requirements  for  the  program? 

•  Has  the  need  to  update  training  with  product  changes  been  addressed? 

•  Have  sufficient  support  product(s)  and  spares  been  addressed  for  training? 

■S  Can  commercial  training  meet  program  requirements?  Is  it  available  and  cost-effective? 

■S  Is  the  system’s  OSE  training  requirements  a  key  part  of  the  IPT  design  and  support  considerations  ? 

■S  Have  all  levels  of  support  been  addressed  as  part  of  the  training  plan/effort? 
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4.7. 1  Program  Office/Warfare  Center  Training 

T  he  program  office  and  warfare  center  personnel  can  obtain  overall  open  system  knowledge  in  several 
ways.  One  recommendation  is  to  attend  the  Open  Systems  Joint  Task  Force  (OSJTF)  supported  open  systems 
training  seminars  being  made  available  at  both  the  executive  and  program  office  level.  The  OSJTF  course  is 
available  for  presentation  to  an  entire  program  office  team  (engineers,  acquisition  planning,  and  product  support 
personnel)  and  is  regularly  scheduled  or  arrangements  can  be  made  to  present  it  locally. 

The  Naval  Postgraduate  School  (NPS)  offers  several  curricula  including  Information  Technology 
Management  and  Joint  C3I  curriculum  in  its  Masters  program.  Both  curricula  teach  open  systems  and  introduce 
students  to  standards  in  several  technology  areas.  In  addition  to  the  resident  program,  the  NPS  can  provide  several 
formats  of  outreach  including  resident  short  courses,  seminars,  distant  learning,  and  on-site  short  courses.21 

The  training  costs  of  the  SSA  must  be  considered.  Using  interface  standards  and  languages  common  to 
the  commercial  sector  enhances  the  availability  of  commercial  courses.  One  challenge  is  to  provide  training  that 
keeps  up  with  ongoing  changes  in  commercially  based  products,  such  as  operating  and  data  base  management 
systems. 


These  sources  of  training  are  a  useful  prelude  to  embarking  upon  usage  of  open  system  interface  standards 
in  DoD  systems.  In  addition,  training  on  specific  open  system  interface  standards  is  available  through  standards 
organizations  and  commercial  training  firms. 

4.7.2  Fleet  Training 

To  take  full  advantage  of  open  systems  and  NDI  product  usage  and  support  opportunities,  we  have  to 
overcome  our  cultural  bias  toward  fully  organic  product  support.  Availability  of  commercially  based  open 
standards  and  the  products  that  implement  them  gives  us  the  opportunity  to  examine  existing  training,  determine 
probable  costs,  and  perhaps  test  existing  training  products  and  services  prior  to  large  scale  expenditures 

As  an  alternative  to  pipeline  training,  we  may  take  advantage  of  commercial  training  classes.  Many  of  the 
classes  used  by  commercial  firms  to  train  their  own  employees  and  dealers  can,  in  whole  or  in  part,  be  used  for 
training.  This  is  particularly  applicable  for  applications  where  the  total  number  of  people  to  be  trained  is  small,  or 
when  the  need  for  training  is  infrequent.  We  would  need  the  ability  to  contract  for  the  training  on  a  timely  basis. 
We  can  pay  the  supplier  to  come  to  us  or  send  the  students  to  the  commercial  or  other  government  training 
location. 


Fleet  personnel  training  locations  may  be  shifted  by  a  combination  of  open  system  usage  and  other 
technology-based  trends.  The  commonality  of  technology  application  across  formerly  disparate  systems,  have 
resulted  in  a  reduction  of  the  number  of  NECs  e.g.,  sonar,  radar,  communications,  command  and  control.  With 
fewer  NECs  to  train,  the  number  of  unique  training  courses  required  to  support  each  system  can  be  reduced. 
Likewise  fewer  instructors  will  be  required  to  support  training  needs.  In  the  sense  that  this  enables  fewer  people  to 
maintain  larger  numbers  of  products  and  systems,  this  is  beneficial.  In  terms  of  the  capabilities  now  expected  of 
maintenance  people,  it  is  a  major  challenge.  One  way  to  meet  this  challenge  is  to  use  electronically  based  training, 
provided  at  the  work  location,  e.g.,  ship,  rather  than  in  formal  pipeline  training  sessions.  Modeling  and  simulation 
are  also  tools  suitable  for  training  maintainers  as  well  as  operators. 

As  open  standards  defined  self  diagnostic  capabilities  are  implemented,  a  relative  decrease  in  the  required 
skills  of  maintenance  personnel  can  be  expected  This  in  turn  shall  reduce  fleet  maintenance  training  requirements. 


-1  Refer  to  Internet  url  http://www.nps.navy.mil  for  additional  information. 
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4.7.3  Intermediate  and  Depot  Training 

The  training  expense  associated  with  intermediate  and  depot  level  repair  will,  in  most  open  system 
interface  standard  product  cases,  become  part  of  the  costs  associated  with  commercial  repair  of  the  item.  From  an 
economic  view,  commercial  training  usually  provides  a  higher  quality  of  training,  lower  training  personnel 
turnover,  and  modern,  well-maintained  equipment.  All  these  can  reduce  cost  per  repair.  For  various  reasons,  most 
of  these  training  factors  are  difficult  to  maintain  in  DoD  depot  and  intermediate  repair  facilities.  Centralization, 
spreading  of  fixed  costs  across  as  many  users  as  possible,  and  elimination  of  costs  are  all  routine  commercial 
economic  goals.  Each  of  these  reduces  costs  to  the  end  user,  be  that  DoD  or  some  other  customer. 

In  those  cases  where  organic  depot  or  intermediate  repair  is  deemed  necessary,  training  costs  may  still  be 
reducible  through  usage  of  commercial  training  of  DoD  maintenance  personnel  by  the  OEM.  Obviously,  this 
situation  is  best  addressed  during  the  market  survey  and  proposal  development  phase  of  the  program  office 
activity. 

4.7.4  Examples  and  Rationale 

Consider  training  in  two  scenarios:  use  of  NDI/commercial  items  and  use  of  DoD  unique  products 
developed  in  accordance  with  open  system  interface  standards. 

In  a  commercial  item(or  other  NDI)  usage  scenario,  product  and  associated  training  have  already  been 
developed  or  are  under  development.  Hence  there  is  opportunity  to  review,  evaluate,  and  perhaps  make  use  of  all 
or  part  of  existing  or  already  planned  training  products  and  services.  All  or  some  of  the  products  (such  as 
electronically  based  self  taught  training  courses)  may  be  useful,  with  or  without  being  supplemented  or  modified. 
These  costs  can  be  forecasted  and  either  planned  for  or  avoided  up  front  if  review  of  available  training  products 
and  services  is  performed  as  part  of  the  market  survey.  By  factoring  this  information  into  the  life  cycle  cost 
forecasts  for  competing  products,  trade  off  decisions  as  to  which  product  family  is  best  for  meeting  all 
requirements,  performance  as  well  as  supportability  and  cost,  can  be  made. 

In  this  second  scenario,  opportunity  to  become  an  additional  user  of  existing  training  products  and 
services  is  not  available.  Nonetheless,  some  training  cost  avoidance  is  possible.  Often,  open  system  interface 
standards  based  product  development  to  meet  unique  military  requirements  represents  repackaging  or  enhancement 
of  products  developed  for  other  markets.  In  these  cases,  the  training  products  and  services  associated  with  these 
other  products  may  be  modifiable  for  less  cost  than  full  development  would  imply.  It  is  imperative  that  the  ILS 
Manager,  training  development  specialist,  and  cost  analyst  become  familiar  with  the  actual  product  development 
scenario  envisioned  by  the  bidders,  the  associated  product  families,  and  their  existing  training  scenarios,  products, 
and  costs.  Then  comes  the  real  challenge:  determining  the  comparative  anticipated  costs.  The  tools  available  are 
information  of  typical  training  development  costs  and  of  the  commercial  marketplace,  coupled  with  parametric 
analyses.  Again,  this  is  part  of  the  market  survey. 
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5.0  TOOLS,  PROCESSES,  AND  METHODS 

This  section  provides  information  on  how  some  available  acquisition  related  tools  and  processes  can  be 
used  to  create  a  cost  effective  and  best  value  open  system  which  embodies  current  technology  over  the  entire 
service  life.  Listed  below  are  tools  and  processes  that  are  available  to  reduce  program  risk  and  enhance  the 
probability  of  fielding  and  supporting  an  open  systems  product. 

•  Trade-off  Study 

•  Market  Survey 

•  Modeling  and  Simulation 

•  Test  and  Evaluation 

•  Configuration  Management 

•  Warranty 

Each  of  these  tools  are  referenced  by  other  sections  of  this  Guide;  thus,  placing  them  in  the  context  which 
allows  them  to  be  the  most  useful. 

5.1  Trade-Off  Studies 

Trade-off  is  defined  as  “the  determination  of  the  optimum  balance  among  system  characteristics  (cost,  schedule, 
22 .  Trade-off  studies  are  performed  through  out  the  life  cycle  of  a  product.  The  analysis 
is  based  on  two  basic  data  sources:  market  surveys  (section  5.2);  and,  modeling  and  simulation  (section  5.3).  The  market 
survey  provides  data  on  open  system  interface  standards  and  products  available  as  either  non-developmental  items  (NDI) 
or  commercial  items.  Modeling  and  simulation  provides  data  as  to  potential  architectural  solutions  using  the  various 
standards  or  NDI/commercial  item  products.  The  purpose  of  the  trade-off  study  is  to  achieve  a  “best  value”  solution. 

Commercially  accepted  open  system  interface  standards  provide  several  advantages  which  need  to  be 
considered  in  any  trade-off  study.  These  include: 

•  availability  of  a  wider  range  of  developers  for  initial  product  development,  support,  and  upgrade; 

•  availability  of  NDI/commercial  item  products  that  satisfy  ongoing  supply  support,  functionality,  or 
upgrade  requirements  increase  as  compiling  standards,  profiles,  and  products  are  further  refined;  and. 

•  similarity  of  commercial  design  practices  and  products  may  reduce  the  cost  of  developing  technical 
documentation,  training,  and  repair  by  making  them  more  readily  available  and  less  costly  to  provide. 


22  James  V.  Jones,  Integrated  Logistics  Support  Handbook,  TAB  BOOKS  Inc.,  1987. 
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The  purpose  of  trade-off  studies  is  to  maximize  the  open  system  environment  (OSE)  advantages  in  the  new 
acquisition  while  achieving  program  objectives.  Some  trade-off  study  considerations  are  provided  below. 


TRADE-OFF  STUDY  CONSIDERATIONS 


✓ 


✓ 

✓ 


Have  all  of  the  system  performance  requirements  been  achieved? 

•  If  not,  can  the  shortfalls  be  mitigated? 

•  Can  the  system  requirements  be  modified? 

Have  the  system  support  requirements  been  achieved? 

•  Is  a  commercial  support  infrastructure  utilized? 

•  On  a  product  by  product  basis? 

•  Does  an  organic  support  infrastructure  need  to  be  developed? 

In  a  legacy  environment,  consider  partial  or  complete  replacement  of  system  or  sub-systems. 

Pay  to  enhance  support  for  product  “A”  or  use  product  “B”  which  has  full  support  but  less  performance. 
If  a  product  is  modified,  will  the  DoD  be  the  only  user  of  the  resulting  product? 

Can  repackaging  make  a  product  or  interface  feasible  in  the  design? _ 


The  same  considerations  and  questions  should  be  asked  throughout  the  acquisition  program,  especially  when 
evaluating  applications  of  selected  products.  The  program  office  may  not  have  to  upgrade  with  each  next  generation  of  a 
product  to  meet  the  program's  requirements.  However,  the  program  office  should  keep  monitoring  the  products  for 
conformance  to  standards,  functional  features,  and  performance  capabilities.  It  is  essential  that  the  acquisition  and 
logistics  managers  track  the  products  to  ensure  the  fielded  components  have  not  changed  without  notice.  Trade-offs  will 
be  an  ongoing  effort  with  evolving  standards  and  products. 

5.2  Market  Survey 

With  so  many  technologies  and  products  for  widespread  commercial  use  by  standards  initiatives,  technical 
and  system  architectures  and  product  choices  can,  and  must,  be  made  on  the  basis  of  economic  as  well  as  technical 
merit. 


The  market  survey  is  a  critical  tool  in  reducing  programmatic  and  technical  risks.  The  following  table 
consists  of  questions  typically  addressed  during  a  market  survey.  Some  questions  are  more  applicable  to  hardware 
than  software.  Some  have  metrics  which  can  be  assigned  to  some  of  them;  others  do  not.  The  questions,  metrics, 
and  ranking  as  to  importance  should  be  tailored  for  program  usage. 


MAKET  SURVEY  CONSIDERATIONS 


Maturity  of  Standards,  Technologies,  and  Products 

V  How  mature  is  the  technology? 

•  State  of  the  art? 

•  State  of  the  commercial  practice? 

V  Is  the  standard/profile  tight  enough  to  promote  interoperability  of  compliant  products? 

V  Are  the  products  fairly  stable? 

V  What  is  the  product  cycle  time? 

V  When  is  the  next  scheduled  product  update? 

V  Are  the  products  being  refined  or  vastly  changed  at  each  cycle? 

_ Market  Acceptance _ 
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MAKET  SURVEY  CONSIDERATIONS 

Are  the  standard(s),  profile(s),  and  product(s)  well  accepted  in  the  commercial  marketplace? 

•S  What  is  its  market  share? 

•  Is  it  increasing? 

•  Is  it  being  outmoded? 

^  Is  the  commercial  market  big  enough  to  imply  long  term  support  and  upgrade  of  the  item? 

Multiple  Product  Sources 

•S  Are  there  multiple  sources  for  compliant  products? 

Are  the  compliant  products  interchangeable  with  others  that  perform  the  same  functions? 

Are  the  compliant  products  interoperable? 

S  Do  they  accept  data  from  each  other? 

S  Do  they  meet  the  same  performance  requirements? 

Are  the  products  pin-for-pin  interchangeable? 

Have  the  product  interfaces  been  conformance  tested? 

•S  What  were  the  test  results? 

Product  Line  Families 

•S  Are  there  product  families? 

•S  Will  usage  of  a  given  product  commit  us  to  a  product  family? 

•S  Will  such  a  relationship  provide  the  best  value? 

•  Is  the  existing  support  structure  well  suited  to  the  requirements? 

•  Will  it  be  required  to  supplement  or  replace  existing  support  products  (e.g.,  technical  data,  training, 
repair,  upgrade,  etc.)? 

Should  the  product  family  or  the  individual  products  be  approved  for  system  use? 

Test  and  Evaluation 

How  is  the  product  tested? 

•S  Is  there  existing  test  and  evaluation  data? 

•S  What  are  the  existing  test  and  evaluation  parameters? 

Does  the  existing  test  capability  meet  the  needs? 

•S  Does  the  existing  test  capability  test  families  of  products? 

How  much  will  it  cost? 

Have  the  product  interfaces  been  conformance  tested? 

•  Does  a  conformance  test  capability  exist  for  the  product  interfaces? 

•  Is  there  existing  conformance  test  data  or  certification? 

•S  Is  there  existing  reliability  data? 

•S  Will  the  government  have  to  invest  in  test  and  evaluation  facilities? 

•  For  integration  ? 

•  For  ongoing  product  support  and/or  upgrade? 

Does  the  standard(s)  have  a  set  of  conformance  test  requirements/procedures? 

•S  Do  the  conformance  test  procedures  test  for  all  mandatory  requirements? 

S  Do  the  conformance  test  procedures  address  the  optional  and  executable  requirements? 

•S  Does  each  product  have  conformance  test  data  available? 

Are  interoperability  test  requirements  addressed  or  are  they  part  of  conformance  test? 

Technical  Data 

S  Does  the  technical  data  currently  exist? 

•S  Is  the  technical  data  to  be  provided  by  the  various  suppliers  sufficient? 

•S  Is  the  technical  repair  documentation  adequate? 

Is  the  data  in  a  usable  format?  If  not,  what  problems  can  be  foreseen? _ 
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MAKET  SURVEY  CONSIDERATIONS 

^  Is  the  technical  data  available  in  an  electronic  format  (e.g..  Joint  Continuous  Acquisition  and  Life  Support 
(JCALS))? 

•  Who  owns  the  data  rights  at  the  end  of  production? 

•  Will  the  data  be  available  after  production  ? 

•S  What  work-around  is  available? 

Configuration  Management 

•S  Does  a  configuration  management  baseline  exist  currently? 

•  Is  the  data  accessible  on-line? 

•  Are  the  latest  designs  documented  and  available? 

•  Are  the  software  version  description  documents  available? 

•S  Can  it  be  worked  around,  modified  or  supplemented? 

•  By  the  contractor  or  the  government? 

•  What  will  the  cost  be? 

•S  When  will  this  cost  be  borne? 

•S  Does  the  contractor  provide  notification  of  product  changes? 

•  In  time  to  make  life  of  type  buy  as  necessary? 

Availability 

•S  What  is  the  operational  availability  (A0)? 

•S  What  is  the  inherent  availability  (  A,)? 

•S  What  is  the  mean  time  between  failure  (MTBF)? 

•S  What  is  the  mean  time  to  repair  (MTTR)? 

Performance  Monitoring  and  Bit 

S  Is  there  Built-in-Test  (BIT)/Built-in-Test  Equipment  (BITE)? 

Are  the  results  accessible? 

•  What  is  the  BIT  performance? 

•  Are  the  self  test  capabilities  acceptable  from  a  system  level  view? 

•S  Will  it  be  hard  to  reintegrate  when  updates  occur  (not  just  from  an  engineering  view,  but  also  the  support 
products  and  services,  e.g.,  training,  configuration  status  accounting,  and  supply  support)? 

Warranty 

•S  Who  will  administer  the  warranty? 

^  Is  an  extended  warranty  available? 

•S  What  will  the  warranty  cost? 

•S  What  is  the  original  equipment  manufacturer’s  (OEM)  historical  response  to  claims? 

•S  When  does  the  warranty  begin? 

•  On  delivery? 

•  On  installation? 

S  Does  the  contractor  (prime  or  integration)  have  existing  repair  agreements/warranties  with  the  OEMs? 

^  Is  the  warranty(ies)  transferable  to  the  government? 

•S  Are  repair  agreements  available  with  the  OEMs? 

Quality  Assurance 

•S  Is  the  company  ISO  9000  registered  ? 

•  If  not,  does  an  acceptable  quality  process  exist? 

•S  What  policies  are  in  place  for  subcontractors? 

Are  the  subcontractors  ISO  9000  registered? 

•  If  not,  does  an  acceptable  quality  process  exist? 

•S  What  is  the  company's  reputation  for  quality  of  product? 

•  For  quality  of  support? _ 
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MAKET  SURVEY  CONSIDERATIONS 

S  What  is  the  company’s  record  of  prior  performance  on  DoD  contracts? _ 


If  the  market  survey  determines  that  a  NDI  solution  is  unavailable,  then  a  development  program  or  some 
mixture  of  development  and  integration  of  commercial  items  will  be  necessary.  Even  in  the  purely  developmental 
situation,  the  selection  process  of  standards  and  profiles  will  drive  system  content  toward  specific  technologies. 
These  technologies  have  associated  NDI  products  (e.g.,  interface  chip  sets)  that  directly  affect  the  cost  and 
supportability  of  the  resulting  system.  These  technologies  and  products  should  be  evaluated  to  determine  their 
supportability  impacts  on  the  system  to  be  developed. 

To  the  extent  that  NDI  and  commercial  items  are  present  within  the  system,  those  items  must  be  evaluated 
in  terms  of  usefulness  in  varied  support  environments  as  listed  below: 

•  System  design  agent  facility  or  development  laboratory. 

•  System  integration  facility. 

•  Test  and  Evaluation  facility. 

•  In-Service  Engineering  Agent/Cognizant  Field  Activity  (ISEA/CFA). 

•  Software  Support  Activity  (SSA). 

•  Installation  and  Checkout. 

•  Training. 

•  Fleet  operation  and  maintenance. 

Support  requirements  for  each  of  these  environments  have  unique  and  overlapping  aspects  which  must  be 
analyzed,  planned  for,  and  implemented.  Based  on  analysis  of  requirements  versus  NDI  support  products  and 
services,  costs  and  schedules  can  be  forecasted  and  compared. 

An  existing  support  product  or  service  (e.g.,  technical  consultation)  may  be  well  suited  to  some  support 
environments  but  ill  suited  to  others.  For  example,  technical  consultation  services  needed  to  enable  the  design 
agent  to  successfully  utilize  a  product  when  designing  a  system  are  generally  inappropriate  to  support  the  same 
product  in  a  training  or  Fleet  environment.  The  types  of  questions  asked  as  well  as  the  goals  and  pre-existing 
knowledge  base  of  both  the  people  asking  and  answering  the  questions  are  different.  The  documentation  required 
probably  overlaps,  but  is  not  the  same. 

The  role  of  the  supportability  members  of  the  market  survey  team  is  to  determine  what  product  support 
products  (e.g.,  spares  and  technical  manuals)  and  product  support  services  (e.g.,  technical  consultations  and 
hardware  repair)  are  available.  Each  must  be  evaluated  against  the  support  requirements  of  the  environments 
listed  above.  Every  program's  needs  and  opportunities  will  be  unique. 

For  each  requirement,  available  NDI/commercial  item  products  and  services  should  be  identified, 
analyzed  and  evaluated.  Where  existing  support  products  and  services  meet  the  requirements,  anticipated  usage 
scenarios  and  costs  can  be  identified.  When  available  NDI/commercial  item  support  products  and  services  will  not 
meet  requirements,  the  sources,  schedules,  and  costs  to  supplement  or  develop  future  updates  must  be  estimated. 
In  most  cases,  estimated  support  costs  for  each  product  being  considered  can  be  determined  and  compared  to  that 
of  its  competitors.  This  information  can  generally  be  estimated  based  on  inputs  from  the  OEM  and  present 
customers.  Thus  support  costs  of  different  products  can  be  used  as  a  driving  factor  toward  architectural  choices 
based  on  an  overall  best  value  prior  to  commitment  to  a  particular  design  architecture  or  product  baseline.  The  pay 
off  for  the  time  and  staff  utilized  for  this  effort  is  a  best  value  system,  an  executable  defensible  program  plan,  and  a 
solid  basis  for  a  Planning,  Programming,  and  Budgeting  System  submission. 
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Personnel  participating  in  the  market  survey  can  be  drawn  from  the  program  office,  ISEA/CFA,  SSA,  and 
other  interested  groups  who  know  the  usage  environments  and  expectations.  These  personnel  should  have  a  stake 
in  ensuring  long  term  success  of  the  effort. 

Figure  5-1  provides  a  sample  market  survey  format  used  for  product  evaluation.  However,  further 
evaluation  and  testing  of  the  product  should  be  performed  to  ensure  it  meets  the  open  system  interface  standard 
profile.  Considerations  such  as  software  maintenance,  available  product  data,  software  configuration  management 
processes,  supportability,  schedules,  and  other  key  issues  as  identified  in  the  market  survey  form  need  to  be 
evaluated. 
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j  PRODUCT  SURVEY  FORM  | 

|  PRODUCT  | 

REQUIRED  HARDWARE  ENVIRONMENT 

LICENSING  AGREEMENT 

|  RELEASE:  i 

□  IBM  PC  COMPATIBLE  □  DEC 

|  VERSION:  | 

■  INI 'll  ll  ^^^1 

PRODUCT  APPLICATION  AREAS 

WARRANTY(IES) 

□  SYSTEM  SIMULATION  □  REQUIREMENTS  TRACE 

□  SUN-3,  4  SPARC  STATION  □  HP  APOLLO 

□  REQUIREMENTS  ANALYSIS  □  TESTING 

□  HP  9000  □  OTHER  (SPECIFY) 

□  COMMUNICATIONS  □  QUALITY  ASSURANCE 

□  APPLE  MACINTOSH 

PRICE/COSTS 

□  LOGISTICS  □  DATABASES 

MEMORY  REQUIREMENTS  HARD  DISK: 

PRICE:  $ 

MAINTENANCE:  $ 

□  DOCUMENTATION  □  DESIGN 

RAM: 

UPGRADE:  $ 

SERVICE  CONTRACT:  $ 

□  METRICS  □  PROJECT  MANAGEMENT 

I/O  DEVICE  REQUIREMENTS: 

QUANTITY  BUY:  $  /  j 

□  SOFTWARE  ENGINEERING  □  MODELING 

REQUIRED  SOFTWARE  ENVIRONMENT 

|  USERS  OF  PRODUCT  [ 

VENDOR 

□  MS-DOS/PC-DOS  | 

□  SUN/OS 

□  VMS 

|  ADDRESS:  | 

PHONE: 

□  UNIX  SYSTEM  V 

□  OTHER  (SPECIFY) 

FAX: 

□  MICROSOFT  WINDOWS  SPECIFY 

□  IBM  MVS,  VM 

INSTALLATION 

DEVELOPER  DATA 

□  HP-UX  | 

□  OTHER  (SPECIFY) 

COMPLEXITY  (1-101  |  CONFIGURABILITY  (1-10) 

SIZE  OF  INSTALLED  BASE: 

□  POSIX 

FINANCIAL  INFO: 

NETWORK(S)  SUPPORTED?  □  YES  □  NO 

1  1 

IF  YES,  SPECIFY: 

MARKET  SHARE:  % 

DATA  TYPE  AND  FORMAT 

PRODUCT  OVERVIEW/DESCRIPTION 

|  INPUT:  ! 

|  OUTPUT:  | 

ARCHITECTURE  (PRODUCT  OPENNESS) 

ARCHITECTURE  TYPE: 

1  1 

|  API  READ/WRITE  CAPABILITY:  | 

1  1 

1  1 

|  ADHERENCE  TO  STANDARDS:  (IGOSS,  GOSIP,  etc.)  | 

1  1 

1  1 

|  DATA  FILE  FORMAT  AVAILABLE/DOCUMENTED:  | 

1  1 

1  1 

PRODUCT/VENDOR  SUPPORT  ] 

1  1 

□  VENDOR  SUPPLIED  TRAINING  □  TELEPHONE  HELP  LINE 

□  ON-SITE  TRAINING  □  USE  GROUP  ORGANIZED 

□  DEMO  DISK(S)  □  ON-SITE  DEMO 

□  OFF-SITE  TRAINING  □  NEWSLETTER  AVAILABLE 

□  LITERATURE  □  TRIAL  PACKAGE 

□  VIDEO  TRAINING  □  ELECTRONIC  BULLETIN 

□  COMPUTER-BASED  TRAINING  □  CONFERENCES 

|  □  USER  EVALUATIONS  □  OTHER  SPECIFY  | 

□  SOURCE  CODE  AVAILABLE  □  S/W  FIX  BETWEEN 

RELEASES 

PERFORMED  BY: 

AGENCY  PERFORMED  FOR: 

□  VENDOR  CUSTOMIZING  □  S/W  UPGRADES 

□  CUSTOMER  CUSTOMIZING  □  DOCUMENTATION 

UPGRADES 

□  COPY  DOCUMENTATION  □  SITE  LICENSE  AVAILABLE 

□  EMBEDDED  COPY  PROTECT  □  NETWORK  VERSION 

TELEPHONE: 

□  BACKUP  COPIES  ALLOWED  □  WINDOW  USER 

INTERFACE 

□  INTERACTIVE  DEBUT  □  GRAPHIC  USER 

INTERFACE 

EMAIL: 

DATE: 

Figure  5-1  Sample:  Market  Survey  Form 
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5.3  Modeling  and  Simulation 

The  need  to  speed  the  acquisition  process,  yet  ensure  reliable  and  supportable  products  are  being  fielded  at 
best  value,  is  at  least  partially  met  through  use  of  modeling  and  simulation  (M&S).  M&S  products  and  approaches 
have  been  used  by  the  DoD  for  some  time.  Historically,  M&S  has  been  expensive,  requiring  a  large  and  highly 
technical  staff  to  first  execute  and  then  analyze  and  validate  the  results.  An  example  is  VHSIC  Hardware 
Description  Language  (VHDL),  used  to  simulate  circuit  board  design  and  performance.  DoD  is  now  emphasizing 
use  of  M&S  as  a  means  to  reduce  time  and  cost  of  complex  acquisition  programs.  The  DoD  plan  for  upgrading  the 
process  within  DoD  is  provided  in  DoD  5000.59P,  Modeling  and  Simulation  (M&S)  Master  Plan,  dated  October 
1995. 


Faster  and  more  complex  computers  have  been  fielded  along  with  better  software  products  allowing 
industry  to  apply  these  technological  features  to  create  M&S  tools  that  are  state-of-the-art.  These  tools  can  assist 
the  program  office  to  rapidly  design  and  field  products,  often  within  1.5  to  2  years  from  concept  definition. 
Concurrent  engineering  goes  hand-in-hand  with  the  new  M&S  tools  and  the  personnel  who  apply  them. 

Examples  of  how  M&S  may  be  used  in  the  acquisition  process  to  assess  and  resolve  operational,  system, 
and  technical  requirements  are  provided  below. 


OPERATIONAL 

SYSTEMS 

TECHNICAL 

ASSESSMENTS 

ASSESSMENTS 

ASSESSMENTS 

•  Operational  Connectivity 

• 

Performance 

• 

Environmental  Considerations 

•  Wargaming  Assessment 

• 

Inter/Intra-Connectivity 

• 

Performance 

-  Joint 

• 

Interface  Applications 

• 

Design  Feasibility 

-  Service 

-  Joint 

• 

Interface  Standards  Considerations 

•  Logistics  (Operational  Support) 

-  Service 

-  Compliance 

•  Training 

-  Changes/Impacts 

-  Conformance 

• 

Throughput 

-  Impacts 

• 

Contentions 

• 

Form,  Fit,  Functionality 

• 

Training 

• 

Training 

• 

Logistics 

• 

LCC 

• 

Life  Cycle  Cost  (LCC) 

• 

Cost  Realism 

• 

Changes/Impacts 

M&S  is  used  to  model  proposed  interface  standards  and  waveforms/protocols  to  determine  interoperability 
factors  and  identify  impacts,  thus  lessening  the  risks  to  go  to  production.  Not  only  can  M&S  can  be  used  to  verify 
design  consideration,  but  for  DoD,  it  can  be  used  early  in  an  acquisition  life  cycle  to  do  trade-off  in  terms  of 
operational  and  system  architecture  impacts,  especially  if  the  target  architectures  have  been  baselined.  M&S  of 
communication  network  interface  standards  can  identify  connectivity  capabilities  and  shortfalls,  as  well  as  impacts 
to  architectural  objectives. 

M&S  can  be  used  to  assess  the  feasibility  of  a  design  prior  to  prototyping.  Modeling  the  design  prior  to 
prototyping  can  identify  functional  and  physical  disconnects,  assess  interoperability  prior  to  building  engineering 
development  models  or  prototypes.  M&S  can  simulate  platform-to-platform  configurations,  cable  installations, 
electromagnetic  compatibility,  and  environmental  considerations  for  platform  type(s),  etc.  Additionally,  the  use  of 
systems  such  as  JCALS  can  help  in  timely  distribution  of  the  critical  M&S  data  to  those  who  need  it..  Figure  5-2 
provides  examples  of  typical  benefits  that  a  program  office  can  achieve  using  M&S  in  the  acquisition  of  OSE 
systems. 
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CONNECTIVITY 

•  TIMING 

•  PROTOCOLS 

•  INTERFACE  COMMONALITY 

•  TRANSMISSION  MODES 

•  CONTENTIONS 

•  DATA  RATES 


INTERFACE/SB  A  COMPLIANCE 

•  INTERFACE  STANDARDS  CONFORMANCE 

•  PROTOCOLS 

•  MSG  FORMATS 

•  DATA  FORMATS 

•  HCI  COMPLIANCE 

•  JTA  COMPLIANCE 


PERFORMANCE 

•  THROUGHPUT 

•  PROCESSING 

•  PERFORMANCE 

•  ACCURACY 

•  PRECISION 

•  DATA  REQUIRED 

•  DATA  GENERATED 


CM 

•  BASELINE  CONTROL 

•  CHANGE  ANALYSIS 

•  FORM,  FIT,  FUNCTION 

•  INTERFACE  COMPLIANCE 


SECURITY 

fl 

•  COMPLIANCE 

•  APPLICATION 

•  ENCRYPTION  /DECRYPTION 
ASSESSMENT 

•  FEA  ASSESSMENTS 

LOGISTICS 

•  LCC  DATA 

•  RMA  ANALYSIS 

•  INSTALLATION 

•  TRAINING 


PROGRAMMATIC 

•  PROGRAM  MANAGEMENT 

•  SCHEDULING/IMPACTS 

•  VALIDATION  OF  REQUIREMENTS 

•  T&E  PROCEDURE  ASSESSMENTS 


Figure  5-2  Typical  Benefits  From  Applying  Modeling  &  Simulation 


5.4  Test  and  Evaluation 

Open  system  interface  standards  foster  use  of  rapidly  changing  hardware  and  software  in  DoD  systems. 
The  use  of  immature  state  of  the  art  products  and/or  products  that  don’t  have  underlying  test  data  can  result  in 
increased  testing  needs.  Strictly  speaking,  one  could  say  that  it  is  not  open  system  interfaces,  as  such,  that  drive 
this  need.  If  interfaces  are  well-defined,  remain  stable,  and  are  adhered  to  scrupulously,  products  that  make  up 
higher  assemblies  and  systems  can  be  expected  to  work  together.  Costs  associated  with  integration  and  operational 
testing  would  not  be  driven  by  interfaces.  Unfortunately,  it  is  quite  common  to  encounter  products  which 
implement  some  but  not  all  features  (e.g.,  services)  specified  by  a  given  standard  and  standard  profile.  Knowing 
which  features  within  an  interface  standard  and  standard  profile  (mandatory  and  optional)  are  actually  used  is 
important.  Equally  important  is  whether  vendor  extension  features  (features  defined  outside  the  scope  of  a 
standard)  are  utilized  (caution:  allowing  the  use  of  vendor  extensions  often  leads  to  a  sole  source  relationship). 
Product  choices  as  well  as  test  planning  and  costs,  will  be  affected.  When  processor,  data  transfer  rate, 
computation  or  some  other  design  factor  affecting  performance  is  changed,  testing  is  needed  to  affirm  the 
performance  of  the  new  item  with  other  system  components  as  well  as  it’s  conforming  to  interface  specifications. 

Choices  of  standards,  technology,  and  actual  products  will  impact  test  and  evaluation  (T&E)  requirements 
(e.g.,  facilities  and  personnel)  and  cost.  These  costs  should  influence  product  usage  decisions.  See  T&E 
considerations  in  the  market  survey  section  5.2. 
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5.4.1  Testing  Strategies 

Strategies  for  test  and  evaluation  of  a  standards-based  product  are  an  important  acquisition  concern. 
Conformance,  interoperability,  and  performance  testing  provide  valuable  information  about  a  product  being 
acquired.  The  resulting  information  will  reduce  risk.  Interface  standards,  profiles,  and  application  product 
interfaces  (APIs)  must  be  exactly  specified,  or  integration  and  other  test  (performance,  conformance)  costs  will  be 
dramatically  increased.  Products  which  do  not  meet  advertised  conformance  and  performance  parameters  cause 
integration  and  test  to  become  more  difficult,  time  consuming,  and  costly. 

5.4. 1.1  Conformance  Testing 

Conformance  testing  involves  testing  products  to  the  requirements  of  an  open  system  interface  standard. 
Conformance  testing  leads  to  identification  of  any  non-conformant  behavior,  including  exception  conditions  such 
as  error  detection  and  recovery  as  defined  in  the  standard.  Conformance  testing  itself  uses  conformance  test 
standards  developed  through,  and  approved  by,  independent  standards  bodies  (i.e.,  ISO,  IEEE,  ANSI).  Testing 
methods  and  requirements,  defined  in  conformance  test  standards,  are  implemented  on  test  equipment  resulting  in 
reusable  test  setups  that  provide  repeatable  results.  That  way,  test  results  are  traceable,  via  the  appropriate 
conformance  test  standard,  back  to  open  system  standard  requirements. 

During  testing,  test  software  exercises  key  product  functions,  which  in  turn  exercise  open  system 
interfaces.  Therefore,  during  conformance  testing  key  product  performance  functions  and  the  interfaces  can  be 
tested.  Furthermore,  as  conformance  testing  relies  on  product  documentation  to  setup  and  perform  the  testing, 
documentation  accuracy  is  verified. 

Conformance  testing  influences  acquisition  decisions  by  effectively  screening  potential  products  for  the 
degree  to  which  they  meet  interface  requirements.  The  information  gained  through  conformance  testing  is 
valuable  during  initial  program  development  and  product  support,  as  well  as  during  procurement  of  upgrades  and 
technology  insertions.  When  performing  conformance  testing  of  potential  alternative  products  for  upgrades,  test 
results  can  be  compared  with  those  of  the  original  equipment  or  software.  Direct  comparisons  can  be  made  only  by 
using  the  same  conformance  test  setup  and  identical  application  software  to  execute  functionality  during  the 
testing.  At  that  point  the  degree  of  upgrade  compatibility  with  the  original  equipment  can  be  determined.  Testing 
in  such  a  manner  eases  the  integration  of  alternative  products  and  upgrades  into  a  system. 

When  possible,  program  offices  should  use  open  system  interface  standards  that  have  established 
conformance  requirements,  procedures,  and  test  facilities,  (figure  5-3).  This  eliminates  the  need  to  develop  those 
capabilities  and  the  costs  associated  with  them. 


Standard  Bodies 


Test  Facilities 


Industry  Open 
System  Standards 


Conformance  Test 
Standards 


Executable  Test  Suite 


Reusable  test  suite 
verifying  mechanical, 
electrical,  and  logical 
interfaces 


Figure  5-3  Conformance  Test  Development 
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5. 4. 1.2  Interoperability  Testing 

Interoperability  testing  involves  the  testing  of  two  or  more  interface  connected  products  for  their  ability  to 
work  together.  Therefore,  this  testing  examines  the  operation  of  interconnected  interfaces.  Interoperability  testing 
does  not  necessarily  include  error  handling  and  recovery  tests  nor  does  it  provide  any  indication  of  products'  ability 
to  interoperate  with  products  not  part  of  the  test  set-up.  Interoperability  testing  complements  conformance  testing 
as  a  predecessor  to  performance  testing. 

5.4. 1.3  Performance  Testing 

In  an  open  systems  environment,  performance  testing  includes  parameters  specified  by  the  interface  as 
well  as  those  which  are  not  (e.g.,  execution  speed  for  a  particular  algorithm). 

Interface  performance  criteria  specified  in  a  standard  are  verified  during  performance  testing. 
Performance  criteria  not  specified  in  an  interface  standard  or  specified  as  a  range  of  values  in  an  interface 
standard,  also  are  measured.  All  testing  of  open  system  products  should  be  considered  part  of  the  test  and 
evaluation  of  an  embedded  system  and  therefore,  should  be  included  in  the  respective  test  and  evaluation  plans. 

5.4.2  Testing  Relationships 

The  purpose  of  accurately  defining  the  standard  profile  of  a  particular  standard  is  to  enable  application  of 
that  criteria  for  conformance  and  interoperability  testing  throughout  the  system  life  cycle.  Figure  5-4  illustrates  the 
relationship  of  testing  where  the  program  can: 

•  assess  the  conformance  of  a  product  to  a  specific  standard. 

•  evaluate  interoperability  of  two  or  more  interconnected  product  interfaces, 

•  test  a  product  to  met  specified  performance  requirements,  and 

•  test  whether  the  product  can  meet  integration  requirements. 

Performance  should  be  evaluated  in  terms  of  both  the  standard  interface  and  the  program's  performance 
requirements.  A  product  may  not  meet  all  the  performance  requirements  specified  by  the  standard  or  profile,  but 
still  be  considered  acceptable  for  the  program’s  applications  if  the  parameters  it  does  meet  exceed  specified 
program  objectives.  If  using  the  product  means  that  other  factors  of  consideration  are  not  jeopardized  and  benefits 
can  be  achieved  by  using  the  product,  then  further  testing  and  evaluation  should  proceed.  For  instance,  a  program 
office  might  consider  using  the  product  (that  doesn't  meet  conformance  requirements)  if  it  meets  program: 
performance,  schedule,  quality,  and  reliability  factors;  and.  is  a  leader  in  cost  effectiveness  and  supportability 
issues. 


When  the  program  office  is  planning  for  integration  testing,  sufficient  data  should  be  available  from  the 
other  three  testing  efforts.  This  will  ensure  enough  test  data  and  conformance  test  information  is  on-hand  to 
adequately  plan  and  conduct  integration  testing  efforts  and  assess  the  programs’  overall  integration  risk  factors. 
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Test  Open  System  Product 
and  Application  Integration 


Measure  Interface  Performance 


Evaluate  2  or  more 
Interconnected 
Product  Interfaces 


Measure  of  Compliance  to 
an  Open  System  Standard 


TESTING  FIDELITY :  Conformance  >  Interoperability  >  Performance  >  Integration 


Figure  5-4  Open  Systems  Environment  Related  Testing 


Part  of  the  DISA  and  related  TAFIM  efforts  is  to  identify  standards  and  products  which  meet  this  criteria. 
The  Navy  and  Air  Force  have  similar  on-going  efforts  and  laboratories  to  test  for  conformance  of  some  open 
system  interface  standards  and  products.  As  products  emerge  and  industry  and  DoD  establish  a  relationship  in  this 
area,  more  test  facilities  will  become  available.  Test  centers  for  open  system  interface  standards  such  as  VME,  X- 
OPEN,  FDDI,  and  others  have  been  or  are  being  established. 

Implementation  conformance  refers  to  those  products  which  have  been  verified  and  tested  to  meet  the 
designated  interface  profile.  Conforming  hardware  and  software  implementations  may  include  a  subset  of  features 
of  the  open  system  interface  standard. 

Conformance  test  efforts  should  enable  the  program  office  to  determine  if  an  implementation  embodies 
the  stipulated  requirements  of  the  standard  and  provide  a  metric  to  judge  progress  of  the  program  towards  and 
open  system  design.  A  result  of  conformance  and  interoperability  testing  is  identifying  the  predictability  of 
behavior  and  exception  conditions  of  a  product  allowing  the  program  office  and  integrated  product  team  (IPT)  to 
identify  selected  error  conditions  and  fault  detection/recovery  capabilities.  Reusable  executable  tests  and 
procedures  can  be  developed  and  a  database  for  tracing  test  results  to  individual  product  can  be  maintained.  The 
establishment  of  such  a  database  will  be  important  later  in  a  program’s  life  cycle  when  a  product  line  updates  or 
changes  but  still  claims  conformance  to  requirements. 

5.5  Configuration  Management  (CM) 

CM  efforts  are  affected  by  the  use  of  NDI,  especially  commercial  items,  and  open  system  interface 
standards.  CM,  consisting  of  configuration  identification,  configuration  control,  configuration  status  accounting, 
and  configuration  auditing,  is  essential  for  definition  and  support  of  open  systems.  CM  is  an  important  tool  used  to 
facilitate  acquisition,  lifetime  support,  and  upgrades  to  all  systems. 
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Configuration  management  tasks  are  performed  by  designers,  OEMs,  integration  contractors,  and 
government  program  office  and  support  activities.  A  majority  of  the  support-phase  configuration  management 
workload  falls  on  the  program  office  or  life  cycle  manager  who  must  analyze  needs  and  opportunities  and  must 
decide  which  changes  should  be  made  in  the  system.  Test  and  evaluation,  acquisition,  and  installation  changes 
must  be  coordinated  between  all  affected  systems  and  documentation,  including  those  at  training  activities, 
maintenance  activities,  and  in  storage  throughout  the  life  of  the  system. 

5.5.1  Configuration  Identification 

ISO  10007,  “Quality  Management  -  Guidelines  for  Configuration  Management”,23  defines  configuration 
identification  as  “Activities  comprising  determination  of  the  product  structure,  selection  of  configuration  items, 
documenting  the  configuration  item’s  physical  and  functional  characteristics  including  interfaces  and  subsequent 
changes,  and  allocating  identification  characters  or  numbers  to  the  configuration  items  and  their  documents.” 

It  is  suggested  that  commercial  items  and  NDIs  be  identified  using  OEM  labeling  and  numbering  systems 
whenever  possible.  This  avoids  the  complexity  associated  with  correlating  a  labeling  and  identification  system 
chosen  by  the  program  office,  prime  contractor,  integration  contractors,  etc.  with  the  OEM  systems. 

5.5.2  Configuration  Control 

Commercial  standard  ISO  10007  defines  configuration  control  as  “Activities  comprising  the  control  of 
changes  to  a  configuration  item  after  formal  establishment  of  its  configuration  documents.”  Configuration  control 
is  essential  for  open  systems,  just  as  it  is  for  proprietary  systems. 

The  configuration  control  board  (CCB)  and  interface  control  working  group  functions  apply  and  are 
critical  to  effective  management  of  commercial  items,  NDI,  and  developmental  open  system  interface  standard 
products.  In  both  developmental  and  NDI/commercial  item  usage  scenarios,  interface  profiles  are  a  critical 
management  tool  with  which  to  both  record  and  manage  interfaces. 

The  CCB  is  different  for  open  systems  which  employ  commercial  items  (hardware  and  software)  because 
it  is  the  OEM’s  interpretation  of  the  market,  not  the  government  CCB,  that  determines  which  changes  will  be 
developed  for  those  parts.24  The  CCB's  role  is  then  to  evaluate  and  decide  whether  to  implement  each  change  as  it 
becomes  available  from  the  OEM.  Those  implementation  decisions  are  complicated  by  these  factors: 

•  Many  changes  to  commercial  item  are  “nice  but  not  necessary”  for  military  systems,  and  thus  invite 
rejection  by  the  CCB.  However,  it  takes  only  a  short  series  of  such  rejections  to  produce  a  system  that  is 
dependent  on  antiquated  (by  marketplace  standards)  commercial  items  which  are  no  longer  supported  by 
OEMs. 

•  Only  standard  product  features  should  be  routinely  accepted  for  use.  Only  when  necessary  should  the 
program  office  or  CCB  authorize  “altered  item(s)”  since  this  will  undoubtedly  lead  to  a  sole  source 
proprietary  (or  at  least  a  government  unique)  solution  to  the  problem  that  doesn't  afford  the  government 
the  opportunity  to  leverage  the  commercial  market  for  this  product. 

•  Changes  to  hardware  and  software  commercial  items  and  NDIs  should  be  tested  before  they  are 
considered  by  the  CCB,  to  ensure  that  the  proposed  change  is  fully  functional  and  compatible  with  other 
portions  of  the  system.  The  whole  point  of  integration  testing  is  to  prove  the  functionality  and 


23  Many  configuration  management-related  standards  are  available  and  in  use  by  PMOs.  Although  the  definitions 
in  this  section  come  from  ISO  10007,  the  guidance  is  applicable  to  all  open  systems,  including  those  that  invoke 
MIL-STD-973. 

2 1  Although  the  PMO  may  not  be  able  to  direct  changes  to  COTS  or  other  NDI,  as  a  customer  the  PMO  can  suggest 
changes  to  the  OEM. 
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performance  of  the  proposed  change  prior  to  introduction  to  the  fleet.  Interface  (standards  and  profiles) 
conformance  verification  is  part  of  the  testing.  This  is  particularly  important  if  the  system  contains 
custom-built  items,  or  if  it  contains  older  commercial  items  that  are  no  longer  generally  used  in  the 
commercial  market,  since  compatibility  with  those  items  will  probably  not  have  been  included  in  the 
OEM’s  compatibility  testing.  If  multiple  versions  of  the  system  are  deployed,  compatibility  testing  should 
be  performed  with  every  system  version  that  may  receive  the  change. 

•  Many  changes  to  software,  particularly  commercial  software,  require  associated  upgrades  in  other  items, 
such  as  operating  systems,  memory,  disk  space,  and  processors.  Installation  of  all  of  those  upgrades  must 
be  carefully  coordinated  to  ensure  that  the  overall  system  remains  functional.  An  accurate  configuration 
status  accounting  system  (section  5.5.3)  is  essential  to  achieving  that  coordination. 

Unauthorized  field  upgrades  or  other  changes  are  more  likely  to  occur  with  open  interface  standards  based 
systems.  For  example,  if  a  ship's  crew  uses  version  2.5  of  a  commercial  database  manager  and  version  4.0  is  on 
sale  ashore,  there  is  some  likelihood  that  someone  will  install  the  new  product. 

5.5.3  Configuration  Status  Accounting 

ISO  10007  defines  configuration  status  accounting  as  “Formalized  recording  and  reporting  of  the 
established  configuration  documents,  the  status  of  proposed  changes,  and  the  status  of  the  implementation  of 
approved  changes.” 

Use  of  a  configuration  status  accounting  system  (CSAS)  is  critical  for  ongoing  system  support  and 
upgrade.  A  CSAS  should  not  only  identify  and  track  configuration  items  by  OEM  and  part  number  but  also  by 
form,  fit,  and  function  in  terms  of  a  performance  specification.  Unless  engineering  and  logistic  support  personnel 
know  the  exact  configuration  of  each  fielded  system,  the  ability  to  maintain  or  upgrade  that  system  will  be 
adversely  impacted. 

The  CSAS  provides  the  data  that  enables  the  program  office  to  identify  acceptable  alternate  items  and 
configurations.  For  example: 

•  To  determine  if  a  fielded  system(s)  need  additional  memory  in  order  to  accommodate  a  software  upgrade. 

•  Whether  a  particular  system  can  use  a  500  watt  power  supply  in  place  of  the  existing  300  watt  power 
supply. 

•  How  to  restore  functionality  and  performance  to  a  system  which  has  experienced  failure  of  a  hardware 
item  which  is  no  longer  available. 

Each  of  these  examples  are  based  on  existing  or  new  configuration  baselines  established  through  a  formal  change 
process. 

5.5.4  Configuration  Audits 

ISO  10007  defines  configuration  audit  as  “Examination  to  determine  whether  a  configuration  item 
conforms  to  its  configuration  documents.”  Traditional  audits  are  of  two  types,  functional  configuration  audit 
(FCA)  and  physical  configuration  audit  (PCA). 

The  requirements  for  performing  configuration  audits  are  not  changed  by  an  OSE.  The  level  to  which  the 
audits  are  performed  may  not  go  to  the  piece  part  but  rather  only  verify  to  the  NDI/commercial  item's  (hardware 
and  software)  part  number  and  revision  (version)  for  configuration  items.  Altered  items  would  be  verified  to  the 
part  number  and  revision  and  verify  the  alteration. 
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5.5.5  Commercial  Configuration  Management 

As  part  of  the  acquisition  reform  effort,  DoD  and  commercial  entities  will  shift  from  military  to 
commercial  standards.  User  program  offices  will  have  to  evaluate  the  manufacturer's  implementation  of  CM  and 
determine  during  the  market  survey  (section  5.2)  if  it  meets  the  program  office’s  needs.  If  the  market  survey 
identifies  that  the  manufacturers’  configuration  management  is  inadequate,  consider  the  following: 


ALTERNATIVE  CONSIDERATIONS 

V  Can  the  government  develop  a  work  around  for  configuration  management? 

V  Can  the  manufacturer  modify  or  supplement  its  CM  program  to  satisfy  program  office  requirements? 

•  Is  the  modification  a  no-cost  modification  ? 

•  Will  the  government  be  required  to  pay  an  additional  cost? _ 


5.5.6  CM  by  Form,  Fit,  and  Function 

CM  by  form,  fit,  and  function  is  not  a  new  concept.  In  the  early  1970s,  the  standard  electronic  module 
(SEM)  program  was  founded  on  the  premise  that  a  given  function  could  be  performed  by  the  same  module  in  a 
large  number  of  locations  (rack  or  box,  unit,  and  system  level).  Since  that  time,  technology  has  changed,  bringing 
the  ability  to  package  more  and  more  functionality  on  the  same  amount  of  real  estate.  Today,  we  see  the  same 
principle,  that  a  given  function  can  be  performed  by  the  same  module  in  a  number  of  locations,  applied  at  higher 
levels  of  functionality,  higher  levels  of  integration.  A  good  example  is  the  Navy's  tactical  computers  (TAC) 
program,  whereby  often  times  identical  workstations  can  be  used  in  a  number  of  different  system  applications. 
Industry,  of  course,  has  been  doing  this  for  years.  The  ubiquitous  personal  computer  is  used  for  everything  from 
making  hotel  reservations  to  controlling  inventory  and  manufacturing  processes.  Today,  open  standards  and 
profiles  provide  a  framework  which  gets  us  part  of  the  way  to  achieving  interoperability,  even  interchangeability. 
Ideally,  open  standards  and  profiles  would  get  us  all  the  way,  but,  we  presently  deal  with  incremental  progress 
toward  that  goal.  Families  of  products,  designed  to  meet  form,  fit,  and  function  requirements  (interfaces), 
generally  function  together  to  provide  convenient  pathways  for  system  upgrade  and  maintenance.  In  each  case, 
rules  of  what  works  together  and  what  doesn't  must  be  known  and  adhered  to.  In  each  case,  product  testing  to 
determine  and  develop  approved  configurations,  as  well  as  actual  configuration  records,  are  needed  to  enable  both 
product  support  and  training  of  operator  and  maintenance  personnel. 

Form,  fit,  and  function,  by  themselves  do  not  equate  to  meeting  all  performance  requirements.  For  each 
possible  application  of  the  computer  resource  product,  there  must  be  a  record  of: 

•  which  configuration  items  are  interoperable, 

•  which  configuration  items  are  interchangeable,  and 

•  which  software  configuration  items  are  usable  with  which  hardware  configurations. 

This  information  will  generally,  but  not  always,  be  accumulated  through  testing.  The  need  and  extent  of 
testing  to  be  performed  will  vary.  For  example,  changing  from  one  350  watt  power  supply  to  another  which  is 
identical  in  terms  of  form,  fit,  and  function  may  not  require  any  testing,  unless,  of  course,  the  cooling  fan  supplied 
with  one  of  the  power  supplies  is  less  capable  than  the  fan  supplied  with  the  other,  and  the  unit  in  which  the  power 
supply  is  used  tends  to  run  hot,  or  is  put  in  a  hot  place.  Worse,  two  single  board  computers,  perhaps  from  the  same 
designer  and  manufacturer,  may  have  different  processors,  or  different  amounts  of  onboard  memory.  The 
application  software  might  or  might  not  run  at  the  same  speed,  or  run  at  all.  Information  based  decisions  must  be 
made  as  to  the  level  of  configuration  management  to  apply  to  different  system  components.  The  information  has  to 
be  kept  up  to  date  and  provided  as  technical  data  to  the  people  providing  actual  maintenance,  supply  support, 
technical  manual  development,  upgrade,  and  training.  A  parts  interchangeability  matrix  (PIM)  can  document  this 
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information.  Development  and  maintenance  of  the  PIM  requires  people  and  facilities  (often  the  same  people  and 
facilities  who  comprise  the  ISEA  and  SSA)  to  function  as  an  ongoing  design  agent. 

5.5.7  Commercial  Markets  and  Rapid  Product  Evolution 

Rapid  product  evolution  increases  the  rate  of  change  within  our  systems.  Modern  technology  enables 
short  product  design  to  production  times,  often  followed  by  short  production  runs  prior  to  further  product  changes. 
As  industry  perceives  possible  market  advantage  through  product  evolution,  the  products  change.  Commercial 
market  trends  are  influenced  by  opportunities  to  harness  and  improve  technology,  to  develop  and  improve  products 
for  an  expanding  and  changing  marketplace.  Our  system  design  will,  to  one  extent  or  another,  follow  commercial 
market  trends.  The  higher  the  level  of  integration  represented  by  an  item  (e.g.,  a  chip  versus  a  multi-chip  module 
versus  a  card  level  product  versus  a  box),  the  greater  the  probability  that  some  of  the  components  will  not  be  the 
same  from  one  production  run  to  the  next.  DoD  and  our  suppliers  both  face  this  problem.  The  major  difference  is 
the  level  of  indenture.  The  challenge  is  to  decide,  in  a  timely  manner,  whether  the  same  components  are  still 
available  and  are  they  still  the  best  choice.  These  are  CCB  decisions  resulting  in  product  baseline  changes, 
interchangeability  data  (e.g.,  the  PIM),  and  changes  to  product  support  and  upgrade  planning.  For  each  potential 
change,  the  following  must  be  addressed: 


ALTERNATIVE  CONSIDERATIONS 

V  Is  the  change  needed? 

•S  Is  the  change  avoidable,  and  if  so,  how? 

•  Life-of-type  buys? 

•  Available  substitutes  from  the  same  or  other  sources? 

•  Wait  for  a  later  revision  or  product  (use  existing  inventory  for  support)? 

V  Is  the  change  cost  effective  (consider  the  alternatives)? 


5.5.8  Configuration  Change  in  DoD  Systems 

Reasons  for  configuration  changes  in  government  systems  are  numerous  (figure  5-5).  By  using 
NDI/commercial  items,  the  costs  can  be  spread  over  other  users  of  the  same  product.  At  whatever  level 
NDI/commercial  items  are  purchased,  a  CCB  process  must  be  used  to  decide  on  whether  to  accept  changes  made 
by  others  as  well  as  whether  to  make  a  life-of-type  buy,  freezing  on  the  previous  revision.  Particularly  for  a  large 
system,  the  market  survey,  T&E,  and  CCB  functions  can  easily  become  ongoing  tasks.  Possible  and  acceptable 
baseline  changes  and  the  ongoing  technical  data,  product  support,  and  training  changes  which  follow  them,  imply 
a  constant  level  of  effort.  This  makes  support  and  upgrade  possible. 
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Figure  5-5  CM  Centered  View:  Stimuli,  Actions,  Results 


A  given  functional  baseline  may  be  represented  by  several  product  baselines.  Accurate  tracking  of 
configuration  data  for  rapidly  evolving  multiple  baselines  is  necessary.  Unless  actual  user  system  baseline 
information  (configuration  status  accounting)  is  kept  up  to  date,  problem  duplication/verification  to  enable 
maintenance  support,  upgrade  planning,  and  training  activities  may  become  impossible. 

Overall,  CM  efforts  can  be  expected  to  increase.  Considering  the  cost  of  CM  activities,  sometime  they  can 
be  buried  (hidden)  by  keeping  the  responsibility  for  CM  with  the  prime  or  integration  contractor. 
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5.5.9  Coping  Mechanisms 

The  availability  and  type  of  data  for  CM  can  vary  with  each  procurement  (e.g.,  spares  buy)  and  the  degree  of 
NDI/commercial  item  integration.  System  baseline  control  can  still  be  adequately  maintained,  but  the  capability  to  control 
or  actually  make  changes  to  a  specific  configuration  item  is  not  as  readily  available  to  the  government.  By  applying 
electronic  data  transfer  (e.g.,  JCALS)  and  by  utilizing  computer  aided  design/computer  aided  manufacturing 
(CAD/CAM),  configuration  control,  tracking,  and  updating  drawings  can  easily  be  accomplished,  often  at  lower  cost  than 
in  the  past.  Additional  benefits  include  the  availability  of  current  drawings  and  the  ability  to  transfer  non-proprietary  data 
to  the  program  office.  To  gain  these  benefits,  request  for  proposals  (RFP)  must  require  the  bidders  to  adequately  address 
CM.  As  part  of  an  OSE  CM  process  requirements  and  cost  determination,  some  typical  questions  that  should  be 
addressed  are  provided  below. 


CONFIGURATION  MANAGEMENT  CONSIDERATIONS 

•S  Have  CM  procedures  and  policies  been  developed  and  implemented? 

•S  Are  CM  procedures  and  standards  adequate? 

•S  Has  cost-effectiveness  been  considered  ? 

Are  CM  considerations  being  addressed  during  each  milestone  phase? 

•S  Has  the  legacy  system  been  analyzed  in  terms  of  interfaces? 

•S  Are  CM  considerations  being  addressed  as  part  of  the  market  surveys?  For  example: 

•  Is  the  product  being  evaluated  under  CM  control? 

•  Is  computer-aided  design/  manufacturing  (CAD/CAM)  available  for  current  design? 

•  Is  vendor/producer  change  control  in  place  and  being  implemented? 

•  Is  electronic  data  sharing  (JCALS)  available? 

•S  Does  the  baseline  information  contain  standards  and  standard  profiles? 

•  Are  other  performance  parameters  (e.g.,  Mb,  access  time,  and  processor  speed)  documented? 

•  Is  the  contract  documentation  (including  standard/profiles)  mature,  complete,  and  exact? 

If  prime  or  system  integrator  is  used,  are  mechanisms  in  place  for  CM  control  of  vendors  and/or  subcontractors? 
•S  Are  mechanisms  in  place  to  identify  changes  to  the  program  office  when  changes  occur? 

•S  When  a  change  is  proposed,  are  the  following  asked? 

•  Is  the  change  avoidable,  and  if  so,  how? 

•  Will  a  life-of-type  buy  be  required? 

•  What  will  it  cost? 

•  Are  substitutes  available  from  same  or  other  sources? 

•  Is  the  proposed  change  essential  or  a  nice  to  have  function  ? 

•  Have  form,  fit,  and  function  considerations  of  a  product  been  considered? 

•S  Are  changes  to  standards/profiles  and  products  being  monitored? 

•  Is  planning  in  place  to  gain  advantage  of  changes  to  standards/profiles  and  products? 

•  Is  test  capability  in  place  to  verify  changes? 


Necessary  access  to  design  and  manufacturing  configuration  information  may  be  easy  to  obtain  from  a 
product’s  source,  or  it  may  require  actual  inspection  of  incoming  articles  by  the  ISEA  or  other  support  personnel. 
No  matter  how  the  access  is  provided,  the  decision  process,  CCB  action,  and  documentation,  are  still  prerequisites 
to  being  able  to  provide  effective  product  support. 

A  recommended  practice  is  to  create  and  maintain  a  PIM  (table  5-1)  for  all  hardware  and  software 
replaceable  items.  The  PIM  tells  which  items  will  function  satisfactorily  in  each  specific  configuration  of  the  host 
system.  The  PIM  is  a  CM  document,  used  by  the  support  and  upgrade  practitioners.  The  PIM  documents 
acceptable  configurations  of  a  system.  Some  sort  of  qualification  or  system  testing  is  necessary  as  a  basis  for 
establishing  and  maintaining  the  PIM. 
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The  PIM  is  a  page  based  document,  which  can  also  be  implemented  as  an  electronic  database.  Each  part 
gets  it's  own  page.  For  each  part,  data  fields  interact,  providing  a  picture  of  what  can  be  used  to  replace  the  item, 
and  what  if  any  other  changes  (e.g.,  software)  will  be  needed.  The  data  fields  are  self  explanatory. 


Part  Number 

Interchange 
-able  With 

Usable  With 
Baselines 

Other 

Hardware  Items 
Required 

Applicable 

Software 

Version 

Notes 

123 

a m 

(V)6 

6R1 

Original  baseline 

Technical  Manual  (V)6 

(V)  7 

7R4 

Technical  Manual  (V)6,7 

(V)8 

125- 3 

126- 4 

8R0 

Technical  Manual  (V)8 

Predecessor 

Upgradeable 

To 

123-2,  123-3 

Table  5-1  Parts  Interchangeability  Matrix  Example 


5.6  Warranty 

Although  warranties  are  not  unique  to  the  open  system  architecture  approach  to  obtaining  systems, 
warranties  are  paramount  to  receiving  quality  products  under  a  open  systems  based  contract. 

5.6.1  Commercial  Warranty  Utilization 

Warranty  coverage  is  required  under  10  USC  2403  whenever  unit  cost  is  greater  than  $100,000  or  the 
total  procurement  exceeds  $10,000,000.  The  warranty  assures  the  government  that: 

•  The  item  meets  structural  and  engineering  plans  and  conformance  with  manufacturing  particulars. 

•  The  item  is  free  from  defects  in  material  and  workmanship  at  the  time  of  delivery  to  the  government. 

•  The  operating  capabilities  or  maintenance  and  reliability  characteristics  of  the  item  meet  those  required 
to  fulfill  military  requirements. 

Conformance  to  design  and  freedom  from  defects  are  traditionally  covered  under  the  inspection  clause  of 
the  contract.  The  operating  capabilities  or  maintenance  and  reliability  characteristics  of  the  item  may  be  addressed 
through  either  an  incentive  warranty  or  an  assurance  warranty. 

An  incentive  warranty  provides  incentives  for  the  contractor  to  exceed  minimum  design,  quality,  or 
performance  levels.  For  such  a  warranty,  the  contractor  can  adapt  a  strategy  to  merely  meet  the  minimum 
performance  levels.  However  the  warranty  should  be  structured  so  that  the  risks  of  failing  to  achieve  the  minimum 
levels,  or  the  profit  associated  with  exceeding  those  levels  will  induce  the  contractor  to  exceed  minimum  levels. 
This  type  of  warranty  does  not  necessarily  meet  the  requirements  of  10  USC  2403. 
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An  assurance  warranty  is  the  type  of  warranty  used  when  the  primary  intent  is  to  assure  that  minimum 
design,  quality,  and  performance  levels  are  achieved.  In  an  assurance  type  warranty  the  contractor  is  responsible 
for  damages  or  for  repair  of  items  failing  during  the  period  of  the  warranty.  Assurance  warranty  remedies: 

•  Redesign.  If  a  defect  pertains  to  the  whole  population,  the  warranty  terms  may  stipulate  a  redesign  and 
replacement  of  the  defective  component.  This  is  similar  to  an  automobile  recall. 

•  Price  adjustment.  In  some  cases,  correction  of  a  defect  may  not  be  possible  or  practical,  and  the  only 
remedy  available  may  be  to  "equitably"  adjust  the  contract  price  downward,  in  this  sense,  the  amount  of 
the  adjustment  must  be  commensurate  with  the  damages  suffered  by  the  government.  An  example  of  such 
adjustment  is  the  logistics  support  cost  (LSC)  guarantee  (LSCG).  If  a  measured  LSC  (MLSC)  is  greater 
than  the  corresponding  guaranteed  value  the  contractor  may  have  to  "pay"  all  or  part  of  the  difference 
through  a  downward  adjustment  in  the  contract  price.  On  the  other  hand,  the  contractor  may  share  some 
or  all  of  the  potential  savings  if  the  MLSC  value  is  lower  than  that  guaranteed.  It  should  be  pointed  out 
that  the  term  "equitable  adjustment"  is  relative,  the  contractor's  perception  may  differ  markedly  from  that 
of  the  government.  Conceptually,  the  term  sounds  much  more  benign  than  the  process  proves  in  reality. 

•  Repair  and  replacement.  This  closely  matches  the  common  perception  of  warranty  in  which  a  defect  may 
be  corrected  through  a  repair  or  replacement  action.  Typically,  such  a  remedy  would  be  applied  to  an 
individual-system  defect  as  opposed  to  a  population  defect.  If  the  contractor  performs  the  repair  or 
supplies  the  replacement,  there  is  no  additional  cost  to  the  government;  if  the  government  performs  the 
repair  or  supplies  the  replacement,  it  may  bill  the  contractor.  The  term  "bill  back"  is  used  to  describe  this 
remedy.  The  amount  or  the  method  by  which  the  amount  is  determined  is  generally  specified  in  the 
contract.  Normally  bill  back  amount  cannot  exceed  the  contractor's  normal  repair  and  replacement  costs. 

5.6.2  Warranty  Example 

SPAWAR  is  currently  testing  a  method  for  processing  the  "repair  and  replacement  warranty"  through  the 
supply  system  using  normal  Military  Standard  Requisitioning  and  Issue  Procedures  (MILSTRIP)  and  Military 
Standard  Transaction  Reporting  and  Accounting  Procedures  (MILSTRAP)  procedures  and  administratively 
shifting  the  burden  of  warranty  management  from  the  user  to  the  supply  system.  The  following  conditions  apply: 

•  The  warranty  period  is  for  the  life  of  the  contract,  that  is,  all  items  and  repair  parts  procured  under  the 
contract  have  the  same  warranty  expiration  date. 

•  The  government  contracts  the  right  to  repair  (e.g.  remove  and  replace  failed  components)  the  hardware 
using  government  labor  without  voiding  the  warranty  provisions. 

•  The  contractor  agrees  to  ship  replacement  parts  without  billing  the  government  for  up  to  60  days  while 
awaiting  receipt  of  the  failed  warranty  part. 

•  The  contractor  and  the  government  enter  into  a  just-in-time  (JIT),  electronic  data  interchange  (EDI) 
purchase  order  arrangement  for  supply  support. 

•  The  government  waives  its  right  to  contractor  on-board  repair  for  the  convenience  of  the  government. 
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5.6.3  Warranty  Procedures 

Provided  below  are  warranty  procedures: 

•  The  Navy  Inventory  Control  Point  (NAVICP),  formally  the  Ships  Parts  Control  Center  (SPCC),  assigns  a 
7  Cog  stock  number  to  repair  parts  under  this  contract  requiring  the  failed  part  be  returned. 

•  The  NAVICP  lists  the  vendor  and  contract  number  for  these  items  in  the  master  reparable  items  list  with  a 
note  to  include  the  serial  number  of  the  component  on  the  turn  in  document. 

•  The  NAVICP  establishes  a  standard  price  based  on  the  cost  of  the  item  plus  SPCC  surcharge  and  a  net 
price  based  on  the  SPCC  surcharge  only. 

•  The  requiring  activity,  using  normal  MILSTRIP  procedures,  transmits  a  funded  requisition  to  NAVCIP 
who  procures  the  item  from  the  vendor  using  established  JIT/EDI  procedures  using  net  price  when  the 
failed  item  is  available  to  be  returned  to  the  vendor  and  standard  price  if  unavailable  or  damaged  by  the 
government. 

•  The  requiring  activity  returns  the  failed  item  via  the  advanced  traceability  and  control  (ATAC)  hub  within 
the  60  day  window. 

•  The  vendor  screens  the  material  for  warranty  eligibility  and  reports  exceptions  back  to  the  government  for 
resolution. 

•  The  NAVICP  bills  exceptions  back  to  the  requiring  activity  if  the  exception  was  caused  by  activity 
negligence  or  if  the  requiring  activities  does  not  return  the  failed  warranty  components. 

The  procedures  delineated  above  were  developed  with  certain  criteria  in  mind.  The  criteria  are: 

•  the  warranty  process  needs  to  be  invisible  to  the  user,  meaning  that  no  extra  effort  would  be  required  from 
the  user  to  process  the  items  that  fail  under  the  warranty, 

•  the  supply  system  is  to  be  used  to  replace  the  parts  that  fail  ensuring  historical  data  for  post-warranty 
operations, 

•  the  proper  billing  and  crediting  procedures  need  to  be  included  as  user  incentives  to  take  advantage  of  the 
warranty  and  use  the  NAVICP  for  requisitioning  parts,  and 

•  the  procedures  are  developed  to  utilize  existing  NAVICP  systems,  as  program  changes  are  lengthy  and 
may  impose  additional  burdens  on  the  fleet. 
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APPENDIX  A 

LIST  OF  ACRONYMS 

A, 

Inherent  Availability 

AIS 

Automated  Information  Systems 

ANSI 

American  National  Standards  Institute 

A0 

Operational  Availability 

API 

Application  Program  Interface 

APL 

Allowance  Parts  List 

ASTM 

American  Society  for  Testing  and  Materials 

ATAC 

Advanced  Traceability  and  Control 

ATM 

Asynchronous  Transfer  Mode 

BEA 

Boundary  Element  Analysis 

BIT 

Built-In  Test 

BITE 

Built-In  Test  Equipment 

C/S/A 

CINCs/Services/Agencies 

C3I 

Command,  Control,  Communications,  and  Intelligence 

C4ISR 

Command,  Control,  Communications,  Computer,  Intelligence,  Surveillance,  and 
Reconnaissance 

CAD 

Computer  Aided  Design 

CAIV 

Cost  as  an  Independent  Variable 

CALS 

Continuous  Acquisition  and  Life  Support 

CAM 

Computer  Aided  Manufacturing 

CBT 

Computer  Based  Training 

CCB 

Configuration  Control  Board 

CCITT 

International  Telegraph  and  Telephone  Consultative  Committee 

CDR 

Critical  Design  Review 

CDRL 

Contract  Data  Requirements  List 

CFA 

Cognizant  Field  Activity 

CFS 

Center  for  Standards 

CGM 

Computer  Graphics  Metafile 

CID 

Commercial  Item  Description 

CINC 

Commander  in  Chief 

CISA 

C4ISR  Integration  Support  Activity 

CM 

Configuration  Management 

CMS2 

Programming  Language 

CSAS 

Configuration  Status  Accounting  System 

CSCI 

Computer  Software  Configuration  Items 

DAB 

Defense  Acquisition  Board 

DFAR 

Defense  Federal  Acquisition  Regulation 

DII 

DoD  Information  Infrastructure 

DISA 

Defense  Information  Systems  Agency 

DoD 

Department  of  Defense 

E&MD 

Engineering  and  Manufacturing  Development 

EA 

Evolutionary  Acquisition 

EC/EDI 

Electronic  Commerce,  Electronic  Data  Interchange 

ECP 

Engineering  Change  Proposal 

EDI 

Electronic  Data  Interchange 
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FCA 

Functional  Configuration  Audit 

FEA 

Finite  Element  Analysis 

FIPS 

Federal  Information  Processing  Standard 

GOSIP 

Government  Open  System  Interconnection  Profile 

GOTS 

Government  Off-The-Shelf 

GUI 

Graphical  User  Interface 

HWCI 

Flardware  Configuration  Item 

I/O 

Input/Output 

IAP 

Integrated  Architectures  Panel 

ICP 

Inventory  Control  Points 

IEEE 

Institute  of  Electrical  and  Electronics  Engineers 

IER 

Interface  Exchange  Requirements 

IGES 

Initial  Graphics  Exchange  Specification 

ILS 

Integrated  Logistics  Support 

IPPD 

Integrated  Product  and  Process  Development 

IPT 

Integrated  Product  Team 

ISEA 

In-Service  Engineering  Agent 

ISO 

International  Organization  for  Standards 

IT 

Information  Technology 

ITF 

Integration  Task  Force 

J6 

Joint  Level 

JCALS 

Joint  Continuous  Acquisition  and  Life  Support 

JCS 

Joint  Chief  of  Staff 

JIT 

Just-In-Time 

JMA 

Joint  Mission  Areas 

JROC 

Joint  Requirements  Oversight  Council 

JTA 

Joint  Technical  Architecture 

LAN 

Local  Area  Network 

LCC 

Life  Cycle  Cost 

LOR 

Level  of  Repair 

LORA 

Level  of  Repair  Analysis 

LRU 

Lowest  Replaceable  Unit 

LSA 

Logistics  Support  Analysis 

LSC 

Logistics  Support  Cost 

LSCG 

Logistics  Support  Cost  Guarantee 

M&S 

Modeling  and  Simulation 

MAPP 

Master  Acquisition  Program  Plan 

MILSTRAP 

Military  Standard  Transaction  Reporting  and  Accounting  Procedures 

MILSTRIP 

Military  Standard  Requisitioning  and  Issue  Procedures 

MLSC 

Measured  Logistics  Support  Cost 

MNS 

Mission  Need  Statement 

MOTU 

Mobile  Technical  Unit 

MTBF 

Mean  Time  Between  Failure 

MTTR 

Mean  Time  to  Repair 
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NATO 

NAVICP 

NAVSEA 

NAVSUP 

NDI 

NEC 

NGCR 

NIST 

NMCS 

NPS 

NW 


North  Atlantic  Treaty  Organization 

Navy  Inventory  Control  Point 

Naval  Sea  Systems  Command 

Navy  Supply  Systems  Command 

Non-Developmental  Item 

Navy  Enlisted  Classification 

Next  Generation  Computer  Resources 

National  Institute  of  Standards  and  Technology 

National  Military  Command  System 

Naval  Postgraduate  School 

Network 


OASD 

OEM 

ORD 

OSA 

OSE 

OSJTF 


Office  Assistant  Secretary  Of  Defense 
Original  Equipment  Manufacturer 
Operational  Requirements  Document 
Open  System  Architecture 
Open  Systems  Environment 
Open  Systems  Joint  Task  Force 


PCA 

PDES 

PDR 

PIM 

POE 

POSIX 

PPBS 

PSA 

PTD 


Physical  Configuration  Audit 
Product  Data  Exchange  using  STEP 
Program  Design  Review 
Parts  Interchangeability  Matrix 
Projected  Operational  Environment 
Portable  Operating  System  Interface 
Planning,  Programming,  and  Budgeting  System 
Principal  Staff  Assistant 
Provisioning  Technical  Documentation 


QA  Quality  Assurance 

QM  Quality  Management 


R&D 

RFP 

RMA 

ROC 


Research  and  Development 
Request  for  Proposal 
Reliability/Maintainability/ Availability 
Required  Operational  Capability 


SAA 

SBA 

SCN 

SDR 

SEI 

SEM 

SGML 

SLOC 

SOO 

SOW 

SPCC 

SQL 

SSA 

STEP 

SYSCOM 


Systems  Application  Architecture 
Standards  Based  Architecture 
Specification  Change  Notice 
System  Design  Review 

Software  Engineering  Institute  at  Carnegie-Mellon  University 

Standard  Electronic  Module 

Standard  Generalized  Markup  Language 

Source  Lines  of  Code 

Statement  of  Objectives 

Statement  of  Work 

Ships  Parts  Control  Center 

Structured  Query  Language 

Software  Support  Activity 

Standard  for  the  Exchange  of  Product  Model  Data 
Systems  Command 
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T&E 

Test  and  Evaluation 

TAC 

Tactical  Computers 

TAFIM 

Technical  Architecture  Framework  for  Information  Management 

TCP/IP 

Transmission  Control  Protocol/Internet  Protocol 

TEB 

Technical  Evaluation  Board 

TEMP 

Test  and  Evaluation  Master  Plan 

TPS 

Test  Program  Set 

TRM 

Technical  Reference  Model 

USD(A&T) 

Under  Secretary  of  Defense  for  Acquisition  and  Technology 

VHDL 

VHSIC  Hardware  Description  Language 

VID 

Vendor  Item  Drawings 

VME 

Versa  Module  Europe 

WAN 

Wide  Area  Network 

WWW 

World  Wide  Web 
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